Stochastic Learning Algorithm Research Articles

Age of Information Optimization for Privacy-Preserving Mobile Crowdsensing

Mobile crowdsensing (MCS)-enabled data collection can be implemented in a cost-effective, scalable, and flexible manner. However, joint sensing data freshness and security assurance have not been fully investigated in the current research. To address these two concerns, the potential game and homomorphic encryption-based joint Age of Information (AoI) optimization and privacy-preservation scheme for MCS is put forward in this paper. At first, the AoI minimization and privacy preservation-oriented MCS system framework is established. Then, the AoI-based spectrum access strategies are derived by a potential game in detail, where the stochastic learning algorithm is used to reach the Nash Equilibrium (NE) solution. Next, based on the somewhat homomorphic encryption method, the encrypted sensing data can be submitted to the service provider (SP) for further processing, where the data content can only be known to mobile workers (MWs) and service requester (SR) with permission. Finally, the numerical results show that our proposed MCS system can simultaneously guarantee data freshness and system security at an acceptable cost.

Binary‐Stochasticity‐Enabled Highly Efficient Neuromorphic Deep Learning Achieves Better‐than‐Software Accuracy

In this work, the requirement of using high‐precision (HP) signals is lifted and the circuits for implementing deep learning algorithms in memristor‐based hardware are simplified. The use of HP signals is required by the backpropagation learning algorithm since the gradient descent learning rule relies on the chain product of partial derivatives. However, it is both challenging and biologically implausible to implement such an HP algorithm in noisy and analog memristor‐based hardware systems. Herein, it is demonstrated that the requirement for HP signals handling is not necessary and more efficient deep learning can be achieved when using a binary stochastic learning algorithm. The new algorithm proposed in this work modifies elementary neural network operations, which improves energy efficiency by two orders of magnitude compared to traditional memristor‐based hardware and three orders of magnitude compared to complementary metal–oxide–semiconductor‐based hardware. It also provides better accuracy in pattern recognition tasks than the HP learning algorithm benchmarks.

Service-Aware Resource Orchestration in Ultra-Dense LEO Satellite-Terrestrial Integrated 6G: A Service Function Chain Approach

With the rapid expansion of the scale of deployed low earth orbit (LEO) satellites, the ultra-dense LEO satellite-terrestrial integrated network (LTIN) is envisioned as a promising architecture in the sixth-generation (6G) system to implement seamless connectivity and high-speed data rate service. Especially for ultra-remote real-time services with long transmission distance and high delay requirements, the integrated network can guarantee its end-to-end service continuity. However, many challenges have been posed to the efficient resource orchestration for the service delivery, owing to the large scale, heterogeneity and high mobility of the integrated network. For each service, its data needs to go through a series of on-board processing, before being downloaded to the terrestrial network for further applications. To this end, service function chain (SFC), an ordered concatenation of network functions (NFs), is introduced to support service provision. By allocating the constituent NFs over the LTIN, we propose an efficient multiple service delivery scheme to minimize the overall delivery completion latency, while taking into account resource sharing and competition among multiple SFCs. First, we formulate the multiple SFC embedding problem as a noncooperative game that is further proved as the weighted potential game with at least one Nash equilibrium (NE). With the help of the proposed global coordination mechanism, we design two algorithms to obtain the NE. One is the best response (BR) algorithm with faster convergence, while the other is adaptive play (AP) algorithm with more capacity for best solutions. Then, the stochastic learning (SL) algorithm is proposed to adapt to network dynamics and reduce global information exchange. Finally, extensive simulations validate the convergence and effectiveness of the proposed algorithms.

Learning capacity and function of stochastic reaction networks

Biochemical reaction networks are expected to encode an efficient representation of the function of cells in a variable environment. It is thus important to see how these networks do learn and implement such representations. The first step in this direction is to characterize the function and learning capabilities of basic artificial reaction networks. In this study, we consider multilayer networks of reversible reactions that connect two layers of signal and response species through an intermediate layer of hidden species. We introduce a stochastic learning algorithm that updates the reaction rates based on the correlation values between reaction products and responses. Our findings indicate that the function of networks with random reaction rates, as well as their learning capacity for random signal-response activities, are critically determined by the number of reactants and reaction products. Moreover, the stored patterns exhibit different levels of robustness and qualities as the reaction rates deviate from their optimal values in a stochastic model of defect evolution. These findings can help suggest network modules that are better suited to specific functions, such as amplifiers or dampeners, or to the learning of biologically relevant signal-response activities.

Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling

With the development of federated learning (FL), mobile devices (MDs) are able to train their local models with private data and send them to a central server for aggregation, thereby preventing leakage of sensitive raw data. In this paper, we aim to improve the training performance of FL systems in the context of wireless channels and stochastic energy arrivals of each MD. To this purpose, we dynamically optimize MDs’ transmission power and training task scheduling. We first model this dynamic programming problem as a constrained Markov decision process (CMDP). Due to high dimensions of the proposed CMDP problem, we propose online stochastic learning methods to simplify the CMDP and design online algorithms to obtain an efficient policy for all MDs. Since there are long-term constraints in our CMDP, we utilize a Lagrange multipliers approach to tackle this issue. Furthermore, we prove the convergence of the proposed online stochastic learning algorithm. Numerical results indicate that the proposed algorithms can achieve better performance than the benchmark algorithms.

Experiential and Stochastic Learning Algorithms Based on the Probability of a Fuzzy Event and Modified Fuzzy Metric Distance in Intelligent Robotic Part Micro-Assembly

Experiential and stochastic learning algorithms for improving the time consumed in robot’s part micro-assembly procedures are presented. A comprehensive comparison of the results with other recent algorithms is described. The experiential learning algorithm unites the neighboring sections in a manner that is very similar to the larger united section based on the specified criterion. If the experiential learning algorithm is consecutively repeated, the micro-assembly dynamics analysis chart will be not only minimized in the number of sections, but also formed such that each united section will have one selected optimal plan with the lowest modified fuzzy metric distance within its own section. Thus, if the new input appertaining to the section is merely identified, the final selected plan within the section will execute the related task quickly and accurately thus resulting to a significant improvement in the time spent in the part insertion. Through the stochastic learning algorithm extended from the crisp set domain to the fuzzy set domain, based on the probability of a fuzzy event and the modified fuzzy metric distance, to deal well with the uncertainties arising during the micro-assembly procedure, if one specific plan among the feasible plans is successively selected to solve its confronting micro-assembly problems, the probability that the selected plan will be continuously chosen will increase as a consequence of compensation, namely, a higher value of membership degree is continuously allocated to the selected plan. As a result, the selected control plan with the highest probability of success and the lowest degree of uncertainty acquired by the stochastic learning facilitates the stabilization of the execution phase of the part insertion task. The degree of uncertainty, which is measured by a modified fuzzy metric distance and related to the task execution of the micro-assembly, is used as a criterion to select the most valid plan. The results demonstrate that the micro-assembly stuck problems can be successfully resolved by the experiential and stochastic learning algorithms.

A Stochastic Learning Algorithm for Machine Fault Diagnosis

Industrial big data bring a large number of high-dimensional sample datasets. Although a deep learning network can well mine the internal nonlinear structure of the dataset, the construction of the deep learning model requires a lot of computing time and hardware facilities. At the same time, there are some nonlinear problems such as noise and fluctuation in industrial data, which make the deep architecture extremely complex and the recognition accuracy of the diagnosis model difficult to guarantee. To solve this problem, a new method, named stochastic learning algorithm (SL), is proposed in this paper for dimension reduction. The proposed method consists of three steps: firstly, to increase the computational efficiency of the model, the dimension of the high-dimensional data is reduced by establishing a random matrix; secondly, for enhancing the clustering influence of the sample, the input data are enhanced by feature processing; thirdly, to make the clustering effects more pronounced, the noise and interference of the data need to be processed, and the singularity value denoising method is used to denoise training data and test data. To further prove the superiority of the SL method, we conducted two sets of experiments on the wind turbine gearbox and the benchmark dataset. It can be seen from the experimental results that the SL method not only improves the classification accuracy but also reduces the computational burden.

A multi-layer approach to classify the risk of corrosion in concrete specimens that contain different additives

There are many different ways to estimate whether a concrete structure will be suffering corrosion degradation as experimental processes and numerical estimations relied on theoretical equations. Nevertheless, these estimations require higher certainty and satisfactory estimation accuracy. Deep Learning is causing profound changes in society. It has inspired unprecedented advances in historically challenging problems. This paper proposes a mathematician approach. A Deep Learning model capable of classifying the risk of corrosion on concrete specimens relied on a standard methodology. It employs the Electrical resistivity measurement for achieving this goal. The deep neural network architecture presented in this study displays a merging of activation functions. Also, a regularized method to avoid overfitting problems was implemented. The model comprises different standard tests as input data and classification of the corrosion risk as output. The outcomes depict an accuracy of about 98%, given the nature of the stochastic learning algorithm. Different concrete samples were analyzed in this research in order to provide data variety to the algorithm and accomplish a robust and stable model.

Variant of Data Particle Geometrical Divide for Imbalanced Data Sets Classification by the Example of Occupancy Detection

The history of gravitational classification started in 1977. Over the years, the gravitational approaches have reached many extensions, which were adapted into different classification problems. This article is the next stage of the research concerning the algorithms of creating data particles by their geometrical divide. In the previous analyses it was established that the Geometrical Divide (GD) method outperforms the algorithm creating the data particles based on classes by a compound of 1 ÷ 1 cardinality. This occurs in the process of balanced data sets classification, in which class centroids are close to each other and the groups of objects, described by different labels, overlap. The purpose of the article was to examine the efficiency of the Geometrical Divide method in the unbalanced data sets classification, by the example of real case-occupancy detecting. In addition, in the paper, the concept of the Unequal Geometrical Divide (UGD) was developed. The evaluation of approaches was conducted on 26 unbalanced data sets-16 with the features of Moons and Circles data sets and 10 created based on real occupancy data set. In the experiment, the GD method and its unbalanced variant (UGD) as well as the 1CT1P approach, were compared. Each method was combined with three data particle mass determination algorithms-n-Mass Model (n-MM), Stochastic Learning Algorithm (SLA) and Bath-update Algorithm (BLA). k-fold cross validation method, precision, recall, F-measure, and number of used data particles were applied in the evaluation process. Obtained results showed that the methods based on geometrical divide outperform the 1CT1P approach in the imbalanced data sets classification. The article’s conclusion describes the observations and indicates the potential directions of further research and development of methods, which concern creating the data particle through its geometrical divide.

Rapid Universal Early Screening for Alzheimer's Disease and Related Dementia via Pattern Discovery in Diagnostic History

Alzheimer’s disease (AD) is a progressive, incurable and ultimately fatal neurodegenerative condition. In this study, we introduce the Zero-burden Co-morbid Risk (ZCoR) score to screen for the future risk of AD and related dementia (ADRD) 1 − 10 years before a clinical diagnosis. Requiring no new bloodwork or cognitive tests, ZCoR leverages uncharted comorbidity patterns, to potentially enable near-instantenous universal point-of-care screening of entire patient populations. In validation, ZCoR (n = 729, 018) achieves out-of-sample AUC > 90% for predicting a diagnosis immediately after screening, an AUC > 87% for a diagnosis made one year earlier than in current practice, and maintaining over > 80% AUC for predictions made a decade earlier, irrespective of sex. We achieve high predictability in patient slacking any of the currently suspected risk factors; demonstrating effectiveness in cohorts at higher risk of missed diagnoses. Additionally, ZCoR can target mild cognitive impairment (MCI) with performance at par with questionnaire-based assessments (AUC 88 − 90%), maintaining high effectiveness (AUC ≈ 80%) for predicting impairment up to 3 years into the future. Powered by stochastic learning algorithms that enhance standard machine learning, ZCoR enables discovery in electronic heath record databases, can reduce ADRD and MCI diagnostic delays, and the impact of socio-economic and demographic variables, with immediate impact on patient outcomes.

Roth-Erev Reinforcement Learning Approach for Smart Generator Bidding towards Long Term Electricity Market Operation Using Agent Based Dynamic Modeling

This article focuses towards agent based implementation of restructured power market with learning capabilities for generators. The market model considered for the analysis is wholesale market and through learning capability of the generator will confront self-sufficient smart generator to perform optimal bidding for a long term. The Agent Based Modeling for Electricity Systems (AMES) permits dynamic testing with learning traders. The whole electricity market is managed by the Independent System Operator (ISO) which computes the hourly Locational Marginal Price (LMP) and commitments of power exchange for day-ahead market operation. The bidding strategy of generators is trained using stochastic reinforcement learning algorithm (JReLM) developed under Java platform. For the analysis, agents are classified as market traders and Independent System Operator (ISO) linking to the IEEE 5-Bus system. The analysis is further extended to an IEEE-30 Bus system and the results demonstrate great potential of the agent based computational ability in the electricity market to help the generators to exercise more market power with optimal bidding than normal generators.

Interference Mitigation for Coexisting Wireless Body Area Networks: Distributed Learning Solutions

When multiple wireless body area networks (WBANs) exist in close proximity to each other, the inter-user interference considerably degrades the signal to interference plus noise ratio of the packets arriving at each WBAN coordinator. Also, the propagation paths within each WBAN experience fading due to the continuous changes in the body posture and mobility of the human body. The most preferred coexisting mechanisms specified in the IEEE 802.15.6 standard is the channel hopping mechanism, which fails to consider the varying radio environment and obtained reward in its channel selection. Thus, our paper investigates this channel selection problem for interference mitigation in a time-varying environment. We formulate this channel selection problem as a finite repeated potential game and propose two learning algorithms, Stochastic Learning Algorithm (SLA) and Stochastic Estimator Learning Algorithm (SELA) to achieve the Nash Equilibrium (NE) of the game. Numerical results show the convergence of the learning algorithms to the NE point of the game. The performance evaluation and impact of parameters on these two algorithms are also analyzed in our paper.

Dynamic Virtual Resource Allocation for 5G and Beyond Network Slicing

The fifth generation and beyond wireless communication will support vastly heterogeneous services and user demands such as massive connection, low latency and high transmission rate. Network slicing has been envisaged as an efficient technology to meet these diverse demands. In this paper, we propose a dynamic virtual resources allocation scheme based on the radio access network (RAN) slicing for uplink communications to ensure the quality-of-service (QoS). To maximum the weighted-sum transmission rate performance under delay constraint, formulate a joint optimization problem of subchannel allocation and power control as an infinite-horizon average-reward constrained Markov decision process (CMDP) problem. Based on the equivalent Bellman equation, the optimal control policy is first derived by the value iteration algorithm. However, the optimal policy suffers from the widely known curse-of-dimensionality problem. To address this problem, the linear value function approximation (approximate dynamic programming) is adopted. Then, the subchannel allocation Q-factor is decomposed into the per-slice Q-factor. Furthermore, the Q-factor and Lagrangian multipliers are updated by the use of an online stochastic learning algorithm. Finally, simulation results reveal that the proposed algorithm can meet the delay requirements and improve the user transmission rate compared with baseline schemes.

Bloodwork-Free Early Prediction of Gestational Diabetes <i>via</i> Pattern Discovery in Past Medical Encounters

Gestational diabetes (GDM) is associated with adverse pregnancy and neonatal outcomes, increased risk for type 2 diabetes (T2D) and other long-term consequences for both the mother and the offspring. GDM is usually diagnosed around 24-28 weeks of gestation via a complicated sequence of glucose challenge tests (GCT) and repeated blood-work, contributing to increased costs and patient/provider burden. In this study we design a stochastic learning algorithm for GDM prediction with dirty, unprocessed raw data comprising only the records of diagnostic codes generated during past medical encounters from a national US insurance claims database. In computing the Gestational-diabetes Comorbid Risk (GCoR) score, we use no laboratory test results, no medications, demographic, or familial information, and yet achieve a sensitivity greater than 83% at 95% specificity with the corresponding area under the receiver operating characteristic curve (AUC) of 96.87% and a positive predictive value (PPV) >53% at the first prenatal visit for low-risk patients (n=648,784). The AUCs when evaluated one, two and four months before the first prenatal visit is respectively 92.75%, 91.82% and 89.97% for the same cohort. For a general cohort with potentially high risk patients included (n=670,417) we achieve AUCs of 95.42% at the first prenatal visit, degrading to 89.24%, 88.06% and 86.08% at the subsequent time points. And for a high-risk cohort (n=104,946), we achieve AUCs of 94.83%, 89.31%, 87.51%, and 85.80% respectively. We significantly outperform reported tools, achieving a over 200% improvement in sensitivity at 95% specificity for the low risk cohort. Additionally, accurate GDM risk assessment months before pregnancy opens new intervention possibilities, including risk management through lifestyle changes, as well as delaying pregnancy briefly to reduce GDM risk. Additionally, subtle differences in comorbidity patterns suggest the possibility of reducing misdiagnosis of disorders that mimic GDM symptomology such as Cushing's disease, and tumors of the adrenal gland. Funding Statement: This work is funded in part by the Defense Advanced Research Projects Agency (DARPA) project #FP070943- 01-PR. The claims made in this study do not reflect the position or the policy of the US Government. DARPA had no role in the authors’ decision to pursue this specific research problem, or authoring this specific article. Declaration of Interests: Authors have no known conflict of interest. Ethics Approval Statement: The de-identified insurance claims used in this study was deemed exempt by the relevant Institutional Review Board.

Dynamic Link Selection for Interference Mitigation in D2D Multicast Content Sharing Networks

In this letter, we study the link selection problem for interference mitigation under dynamic channel conditions in D2D multicast content sharing networks. We firstly transform the problem into a stochastic game, in which the utility of each player (i.e., cluster head) is designed as the maximum expected weighted interference suffered by the receivers in the cluster. Then, we propose a simple and uncoupled link selection algorithm based on stochastic learning algorithm to obtain the Nash Equilibrium points. Importantly, the proposed algorithm only requires little intra-cluster information feedback. Theoretical analyses and numerical results validate the effectiveness of our proposed algorithm under dynamic channel conditions.

Learning-Based User Clustering and Link Allocation for Content Recommendation Based on D2D Multicast Communications

Content recommendation based on device-to-device (D2D) multicast communications is expected to become a promising approach to improve local area services. Importantly, two main challenges should be considered: i) user clustering—in order to be tailored to recommend contents, the members in the same cluster should have great similarity in multiple characteristics, and ii) link allocation—we should use as little resource consumption and information exchange as possible while keeping the recommendation accuracy. In this paper, we firstly quantify the degree of the similarity between two target users with regard to multiple characteristics. Guided by such similarity, we define a clustering validity index in terms of between-within proportion (BWP) to characterize the clustering performance. Then, the issue of user clustering is modeled as a sum BWP maximum problem, and a user clustering algorithm based on modified K-means algorithm is designed to solve it in a fast-operating and low-complexity way. After user clustering, we model the issue of link allocation as a weighted aggregate interference minimization problem, and then transform it to an exact potential game. As such, a link allocation algorithm based on stochastic learning algorithm is proposed which helps to obtain the result of this game in a distributed way without complete information. Also, we analyze its convergence and optimality performance. Simulation results demonstrate the effectiveness of our proposed algorithms.

Online distributed stochastic learning algorithm for convex optimization in time-varying directed networks

This paper investigates a online learning optimization problem in a distributed manner, where a set of agents aim to cooperatively minimize the sum of local time-varying cost functions while communications among agents depend on a sequence of time-varying directed graphs. For such a problem, we first propose a online distributed stochastic (sub)gradient push-sum algorithm by utilizing distributed optimization methods and push-sum protocols. Then we analyze the regret bounds for the proposed algorithm on the cases when the cost functions are convex and strongly convex, respectively. The bound on the expected regret for convex functions grows sub-linearly with order of O(T),where T is the time horizon. When the cost functions are strongly convex with Lipschitz gradients, the regret bound has an improved rate with order of O(lnT). Numerical simulations on the localization in wireless sensor networks are used to show the effectiveness of the proposed algorithm.

Stochastic AUC Optimization Algorithms With Linear Convergence

Area under the ROC curve (AUC) is a standard metric that is used to measure classification performance for imbalanced class data. Developing stochastic learning algorithms that maximize AUC over accuracy is of practical interest. However, AUC maximization presents a challenge since the learning objective function is defined over a pair of instances of opposite classes. Existing methods circumvent this issue but with high space and time complexity. From our previous work of redefining AUC optimization as a convex-concave saddle point problem, we propose a new stochastic batch learning algorithm for AUC maximization. The key difference from our previous work is that we assume that the underlying distribution of the data is uniform, and we develop a batch learning algorithm that is a stochastic primal-dual algorithm (SPDAM) that achieves a linear convergence rate. We establish the theoretical convergence of SPDAM with high probability and demonstrate its effectiveness on standard benchmark datasets.

Inherent Stochastic Learning in CMOS-Integrated HfO2 Arrays for Neuromorphic Computing

Based on the inherent stochasticity of CMOS-integrated HfO2-based resistive random access memory (RRAM) devices, a new learning algorithm for neuro-morphic systems is presented. For this purpose, the device-to-device variability of CMOS-integrated 4-kbit 1T-1R arrays is examined. To demonstrate the performance of the stochastic learning algorithm and the potential of RRAM technologies for neuro-morphic systems, a two-layer mixed-signal neural circuit for pattern recognition is implemented and tested with MNIST data.

Dynamic Computation Offloading for Mobile Cloud Computing: A Stochastic Game-Theoretic Approach

Driven by the growing popularity of mobile applications, mobile cloud computing has been envisioned as a promising approach to enhance computation capability of mobile devices and reduce the energy consumptions. In this paper, we investigate the problem of multi-user computation offloading for mobile cloud computing under dynamic environment, wherein mobile users become active or inactive dynamically, and the wireless channels for mobile users to offload computation vary randomly. As mobile users are self-interested and selfish in offloading computation tasks to the mobile cloud, we formulate the mobile users’ offloading decision process under dynamic environment as a stochastic game. We prove that the formulated stochastic game is equivalent to a weighted potential game which has at least one Nash Equilibrium (NE). We quantify the efficiency of the NE, and further propose a multi-agent stochastic learning algorithm to reach the NE with a guaranteed convergence rate (which is also analytically derived). Finally, we conduct simulations to validate the effectiveness of the proposed algorithm and evaluate its performance under dynamic environment.