Training Computational Complexity Research Articles

Efficiency optimization methods for stochastic configuration networks

To solve the problems of the low efficiency of parameter allocation and time-consuming computation of output weights in the hidden layers of stochastic configuration networks (SCNs), an optimization method is proposed to improve the SCNs construction efficiency. Firstly, with the increase in the number of hidden layer nodes, the key parameters that determine the strictness of the supervision mechanism are reconstructed to speed up the configuration efficiency of hidden layer input weights and biases. Then, the incremental mechanism of the SCNs are combined with the QR decomposition method, and the output weights are calculated by iteratively updating the transformation matrix, thus reducing the computational complexity of training the SCNs. Finally, the proposed method is validated on four standard datasets and historical data of a municipal solid waste incineration process. The experimental results show that the proposed method improves the efficiency of SCN construction while guaranteeing the prediction accuracy of SCNs model.

A maximum entropy-driven support vector classification model for seismic collapse fragility curves estimation of reinforced concrete frame structures

A novel method called maximum entropy-driven support vector classification (MED-SVC) for estimating the seismic collapse fragility curves of reinforced concrete (RC) frames is proposed in this paper. In comparison to kernel-based learning (KBL) methods, a subset selection process based on the maximum Renyi entropy criterion is embedded into the proposed MED-SVC, which can actively select an informative small-scale subset from a large-scale training dataset. The selected subset can accurately reflect the data distribution of the large-scale training dataset, which can be further used to explicitly establish a high-dimensional feature variable. Due to this, the computational complexity of training the proposed MED-SVC model only depends on the dimension of the feature variable, which can effectively solve the problem related to the low efficacy of training the KBL model in the context of a large-scale training dataset. Meanwhile, the proposed MED-SVC model can also reduce the negative effect of randomness in the subset selection process on predictive performance. The robustness, accuracy, and efficiency of the proposed MED-SVC model are thoroughly validated by comparison with random sampling driven SVC (RSD-SVC), KBL methods, random forest, neural network, and naive bayes based on 33,000 damage data of RC frames under earthquakes. Then, the proposed MED-SVC model is applied to estimate the seismic collapse fragility curve of an RC frame structure outside the aforementioned dataset. The results show that the proposed MED-SVC model can generate seismic collapse fragility curves that agree well with those produced by the finite element method (FEM) and performs more robustly than RSD-SVC.

Elements of the development of the theory of construction of educational and training facilities for complex organizational and technical products

Existing training and training systems use tools that can be implemented both as computer training complexes and operational simulators using simulation. Currently, training tools are used that can use mathematical and scenario models of the behavior of simulated objects. The problem of creating a multi-agent distributed training and training system is relevant. The lack of a theoretical justification for this process does not allow for the rapid addition of new elements within the framework of simulations. The novelty of the proposed approach is the decomposition of the model of the modeling object in order to ensure an operational change in the composition of the elements in the modeling. The theoretical justification of the process of building educational and training facilities for complex organizational and technical products will allow us to predict the image of a promising educational and training facility with an imitation of the situation as close as possible to the real one. The problem of creating a multi-agent distributed training and training system is relevant. Currently, existing training and training systems use tools that can be implemented both as computer training complexes and operational simulators using simulation modeling. The lack of a theoretical justification for this process does not allow for the rapid addition of new elements within the framework of simulations. Theoretical substantiation of the process of building educational and training facilities for complex organizational and technical products to ensure operational changes in the composition of elements in the modeling environment. The trends in the construction of educational and training tools for complex organizational and technical products are considered. The requirements for the construction of promising training and training systems have been formed. The implementation of a distributed training simulator system will increase the level of staff training through the use of mathematical and scenario models of the behavior of simulated objects.

Mathematical Formulation of Learning and Its Computational Complexity for Transformers’ Layers

Transformers are the cornerstone of natural language processing and other much more complicated sequential modelling tasks. The training of these models, however, requires an enormous number of computations, with substantial economic and environmental impacts. An accurate estimation of the computational complexity of training would allow us to be aware in advance about the associated latency and energy consumption. Furthermore, with the advent of forward learning workloads, an estimation of the computational complexity of such neural network topologies is required in order to reliably compare backpropagation with these advanced learning procedures. This work describes a mathematical approach, independent from the deployment on a specific target, for estimating the complexity of training a transformer model. Hence, the equations used during backpropagation and forward learning algorithms are derived for each layer and their complexity is expressed in the form of MACCs and FLOPs. By adding all of these together accordingly to their embodiment into a complete topology and the learning rule taken into account, the total complexity of the desired transformer workload can be estimated.

Design of P-FLANN Model for Intelligent Water Fountain Sound Pleasantness Monitoring Using Bio-inspired Computing and Human Speech Perception

Cognitive-inspired Computational Computing systems play a crucial role in designing intelligent health monitoring systems which help both patients and hospitals. It also helps in early and consistent decision-making for various health issues including human psychological health. Water fountains built in parks and public spaces are used as decorative instruments which not only give appealing visuals but also it provides a relaxing environment to the visitors. These natural sounds have a direct effect on the psychological health of visitors. Very few research works are reported on developing the relationship between water sounds and their corresponding psychological impact. This assessment needs trained manpower and a lot of experimental time which is costly and may not be always available. In this paper to access the human health-friendly water fountain sounds from the pleasantness, a Perceptually Weighted functional link artificial neural network (P-FLANN) model is developed. To reduce the computational complexity of training and for faster convergence, swam intelligence-based optimization algorithm is used for updating the weights. It is observed from the comparative simulation results that the proposed P-FLANN model can effectively perform prediction tasks which is not only cost-effective but also 95% accurate and can play a crucial role in designing human health-friendly water fountains in smart cities.

A competitive learning scheme for deep neural network pattern classifier training

To reduce the computational complexity of training a deep neural network architecture using large data sets of 3D scenes, a competitive learning scheme was devised. The proposed algorithm pits a neural network learning algorithm, in this case the standard Adam optimiser, against an evolutionary algorithm that is used to select the most difficult training examples. The overall scheme is similar to a predator–prey system, where the predator (the neural network) strives to optimise its ability to capture (identify) the prey (the training patterns), and the evolutionary procedure selects the prey that so far evaded capture. As a consequence of the evolutionary process, the neural network is presented only a fraction of the training examples, and the computational complexity of the learning procedure is reduced. Experimental evidence showed that the proposed scheme allows reducing the deep neural network training time on different model sets, sometimes significantly, without affecting the recognition accuracy. The proposed predator–prey scheme is fairly independent of the ANN type and training algorithm employed, and has the potential to be beneficial to a wide range of deep learning applications, where practical implementations are often hindered by the time complexity of the training process.

Automated Diabetic Retinopathy Grading based on the Modified Capsule Network Architecture

The most significant source of eye disease is Diabetic Retinopathy (DR) among individuals who have had diabetes for a long time. Early or timely detection of DR stages is crucial for a better prognosis. The subtle distinction among different DR stages(level 0 to level 4) and many structures of varying shapes and sizes make manual recognition challenging and consume more time. Thus, this work focuses on the automatic grading of the retinal images (L-0 to L-4) based on the DR severity using a modified Capsule Network (Caps Net). The ability of Caps Net to capture spatial information of an object and the probability of some entity's existence in an image makes it more suitable for grading DR stages. This work proposes a hybrid model architecture that contains Caps Net followed by a Support Vector Machine(SVM) at its output layer. The main reason for incorporating an SVM at the output layer of Caps Net is to enhance SVM classification using automatic features extracted significantly but also alleviates the computational complexity of training and testing by the Caps Net model. The experiments were conducted on the APTOS 2019 Blindness Detection Data set. The investigation's outcome substantiates the proposed architecture's efficiency over the existing CNNs and Capsule Networks.

A novel deep policy gradient action quantization for trusted collaborative computation in intelligent vehicle networks

The openness of the intelligent vehicle network makes it easy for selfish or untrustworthy vehicles to maliciously occupy limited resources in the mobile edge network or spread malicious information. However, most of the existing trust models rely on evaluating vehicles or data at the application level. For selfish or forgery attacks in intelligent vehicle networks, we propose a trusted deep reinforcement learning (DRL) cybersecurity approach for computation offloading to evaluate the safety and reliability performance in IoT edge networks, including our intelligent system model and a Deep Policy Gradient Action Quantization (DPGAQ) scheme. By introducing a reputation record table and designing a highly decisive communication trusted computing mode, we can accurately predict the untrusted selfish attack of vehicle in the task offloading of the Internet of things. Furthermore, in the multi-vehicle scenario, because the trusted offloading decision is a mixed integer programming problem, which leads to the dimension explosion of channel state and space, we propose a joint action-value quantization with attention mechanism to approximate the continuous actions values to a limited number of discrete values. Because it is not only inefficient but also unnecessary to generate high-dimensional decision actions in each time frame, we prune the infeasible action decisions by order preserving pruning to reduce the computational complexity of training and achieve efficient training on the premise of ensuring accuracy. To verify the feasibility and effectiveness of our proposed algorithm, millions of channels of edge vehicle networks are used as the input data. The simulation results show that compared with the benchmark trust model, DPGAQ achieves more than 72% reputation level, and improves 11%, 10% and 11% respectively in precision, recall and F-score.

FastDTI: Drug-Target Interaction Prediction using Multimodality and Transformers

Recent advances in machine learning have proved effective in the application of drug discovery by predicting the drugs that are likely to interact with a protein target of a certain disease, leading to prioritizing drug development and re-purposing efforts. State-of-the-art techniques in Drug-Target Interaction (DTI) prediction are often computationally expensive and can only be trained on small specialized datasets. In this paper, we propose a novel architecture, called FastDTI, utilizing pretrained transformers and graph neural networks in a self-supervised manner on large-scale (unlabeled) data, which additionally allows for embedding of multimodal input representations, for both drug and protein properties. Extensive empirical study demonstrates that our approach outperforms state-of-the-art DTI methods on the KIBA benchmark dataset, while greatly improving the computational complexity of training, about 200 times faster, leading to excellent performance results.

The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality

Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-layer ReLU networks with respect to various loss functions. After a brief discussion of other parameters, we focus on analyzing the influence of the dimension d of the training data on the computational complexity. We provide running time lower bounds in terms of W[1]-hardness for parameter d and prove that known brute-force strategies are essentially optimal (assuming the Exponential Time Hypothesis). In comparison with previous work, our results hold for a broad(er) range of loss functions, including lp-loss for all p ∈ [0, ∞]. In particular, we improve a known polynomial-time algorithm for constant d and convex loss functions to a more general class of loss functions, matching our running time lower bounds also in these cases.

DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems

We study the problem of distributed training of neural networks (NNs) on devices with heterogeneous, limited, and time-varying availability of computational resources. We present an adaptive, resource-aware, on-device learning mechanism, DISTREAL, which is able to fully and efficiently utilize the available resources on devices in a distributed manner, increasing the convergence speed. This is achieved with a dropout mechanism that dynamically adjusts the computational complexity of training an NN by randomly dropping filters of convolutional layers of the model. Our main contribution is the introduction of a design space exploration (DSE) technique, which finds Pareto-optimal per-layer dropout vectors with respect to resource requirements and convergence speed of the training. Applying this technique, each device is able to dynamically select the dropout vector that fits its available resource without requiring any assistance from the server. We implement our solution in a federated learning (FL) system, where the availability of computational resources varies both between devices and over time, and show through extensive evaluation that we are able to significantly increase the convergence speed over the state of the art without compromising on the final accuracy.

Classification of objects in images with distortions based on a two-stage topological analysis

The authors propose a method for automatic classification of spatial objects in images under conditions of a limited data set. The stability of the method to distortions appearing in images due to natural phenomena and partial overlap of urban infrastructure objects is investigated. High classification accuracy, when using existing approaches, requires a large training sample, including data sets with distortions, which significantly increases computational complexity. The paper proposes a method for a two-step topological analysis of images. Topological features are initially extracted by analyzing the image in the brightness range from 0 to 255, and then from 255 to 0. These features complement each other and reflect the topological structure of the object. Under certain deformations and distortions, the object preserves its structure in the form of extracted features. The advantage of the method is a small number of patterns, which reduces the computational complexity of training compared to neural networks. The proposed method is investigated and compared with the modern neural network approach. The study was performed on a DOTA dataset (Dataset for Object deTection in Aerial images) containing images of spatial objects of several classes. In the absence of distortion in the image, the neural network approach showed a classification accuracy of over 98 %, while the proposed method achieved about 82 %. Further distortions such as 90 degree rotation, 50 % narrowing and 50 % edge truncation and their combinations were applied in the experiment. The proposed method showed its robustness and outperformed the neural network approach. In the most difficult combination of the test, the decrease in classification accuracy of the neural network was 46 %, while the described method showed 12 %. The proposed method can be applied in cases with a high probability of distortion in the images. Such distortions arise in the field of geoinformatics when analyzing objects of various scales, under different weather conditions, partial overlap of one object with another, in the presence of shadows, etc. It is possible to use the proposed method in vision systems of industrial enterprises for automatic classification of the parts that belong to superimposed objects.

A semi-supervised deep learning based video anomaly detection framework using RGB-D for surveillance of real-world critical environments

Video surveillance has become very important in the current era of smart cities. Large amounts of surveillance cameras are deployed at public and private places for surveillance of infrastructural property and public safety. These surveillance cameras generate huge amount of video data and it is impractical for a human observer to continuously monitor these long-hour videos manually and detect any unwanted or anomalous event. This paper presents a multi-modal semi-supervised deep learning based CNN-BiLSTM autoencoder framework to detect anomalous events in critical surveillance environments like Bank-ATMs. The significance of the framework is that it only requires weakly labelled normal video samples for training. We leverage the power of transfer learning by extracting important video features using a compact pretrained CNN to significantly reduce the computational complexity of training and detection. Moreover, due to the unavailability of any dataset for ATM surveillance in the public domain, we also contributed a unique RGB + D dataset for surveillance of ATMs. The proposed framework is tested on the collected RGB + D dataset and other real-world benchmark video anomaly datasets: Avenue and UCFCrime2Local. Results show that the proposed framework gives competitive results with other state-of-the-art methods and can be applied to both indoor and outdoor environments for detection of anomalies in real-world surveillance sites.

Blessing of dimensionality at the edge and geometry of few-shot learning

In this paper we present theory and algorithms enabling classes of Artificial Intelligence (AI) systems to continuously and incrementally improve with a priori quantifiable guarantees – or more specifically remove classification errors – over time. This is distinct from state-of-the-art machine learning, AI, and software approaches. The theory enables building few-shot AI correction algorithms and provides conditions justifying their successful application. Another feature of this approach is that, in the supervised setting, the computational complexity of training is linear in the number of training samples. At the time of classification, the computational complexity is bounded by few inner product calculations. Moreover, the implementation is shown to be very scalable. This makes it viable for deployment in applications where computational power and memory are limited, such as embedded environments. It enables the possibility for fast on-line optimisation using improved training samples. The approach is based on the concentration of measure effects and stochastic separation theorems and is illustrated with an example on the identification faulty processes in Computer Numerical Control (CNC) milling and with a case study on adaptive removal of false positives in an industrial video surveillance and analytics system.

КОНЦЕПЦИЯ СИНТЕЗА КОМПЬЮТЕРНЫХ ТРЕНАЖЕРНЫХ КОМПЛЕКСОВ ДЛЯ ПОДГОТОВКИ ОПЕРАТОРОВ

The concept of synthesis of computer training complexes for training operators of
 potentially dangerous production facilities, based on the application of the developed software modeling platform, is considered

The concept of building training systems for training operators of liquefied hydrocarbon warehouses

The article deals with the concept of building computer training complexes as one of the effective aids for ensuring the safety of potentially dangerous technological processes which include the storage and transportation of liquefied hydrocarbon gases. The authors have developed and described simulator model of one of the key technological devices for the storage of liquefied hydrocarbons which underlies the proposed software platform for the synthesis of training simulator systems.

A boosted SVM classifier trained by incremental learning and decremental unlearning approach

The Support Vector Machines (SVM) classifier is a margin-based supervised machine learning method used for categorization and classification tasks. A Linear SVM classifier uses linear Kernels while a non-linear SVM classifier adopts non-linear Kernels. The Linear SVM classifier is considered an efficient technique, especially for big datasets of high dimensionality in various applications, such as document categorization, time-series classification, outliers’ detection, to name a few. It is much faster to train a linear SVM classifier than a non-linear SVM classifier. For large-scale datasets with various shapes, configurations, and distributions, the computational complexity of training a non-linear SVM classifier is continuously evolving. Current research methods have introduced various problem formulations, solvers, and strategies for speeding up the training process of a non-linear SVM classifier. However, solving a quadratic programming (QP) problem is still challenging, especially for “Big Data”, which poses a great challenge for traditional methods to train a non-linear classifier efficiently. In this paper, we are proposing a novel boosting algorithm to enhance the performance of weak non-linear SVM classifiers using the notion of incremental learning and decremental unlearning. Experimental results over artificial and real datasets with different sizes, shapes, and distributions show that the proposed ensemble boosting algorithm outperforms the individual SVM classifiers measured by the classification accuracy and the Speedup.

МОДЕЛИРОВАНИЕ ОБРАТНЫХ ПОТОКОВ В ТРЕНАЖЕРНОЙ МОДЕЛИ

Approaches to mathematical modeling of reversible technological flows in training models of computer training complexes are considered.

Система автоматической оценки действий обучаемых в компьютерных тренажерных комплексах

Система автоматической оценки действий обучаемых в компьютерных тренажерных комплексах

Hyperspectral Image Classification With Stacking Spectral Patches and Convolutional Neural Networks

Jointly combining the spatial and spectral features has proved to dramatically improve the performance of classifying hyperspectral image. Recently, utilizing neural networks to automatically model the spatial-spectral feature representations for hyperspectral images has become of great interest. This paper proposes a simple but innovative framework to classify hyperspectral image with two shallow convolutional neural networks (CNNs). First, principal component analysis whitening is applied to decorrelate hundreds of spectral bands. Instead of selecting the principal components to reduce the spectral dimensionality, we retain all the spectral bands but compress the image cuboid into a one-channel spectral quilt by stacking spectral patches . In this way, not only all the spectral information is retained but also the computational complexity of training a neural network is reduced compared with conventional networks that directly input the spectral volumes. Moreover, the spectral quilt will contain some novel textural patterns that are effective at distinguishing classes. Two shallow CNNs are then applied to classify the spectral quilts. As shown in the experiments, both networks can outperform the standard analysis methods.