Stochastic Gradient Descent Algorithm Research Articles

Coherent beam combining of two all-PM thulium-doped fiber chirped pulse amplifiers

In this paper, we report a coherent beam combining (CBC) system that involves two thulium-doped all-polarization maintaining (PM) fiber chirped pulse amplifiers. Through phase-locking the two channels via a fiber stretcher by using the stochastic parallel gradient descent (SPGD) algorithm, a maximum average power of 265 W is obtained, with a CBC efficiency of 81% and a residual phase error of λ/17. After de-chirping by a pair of diffraction gratings, the duration of the combined laser pulse is compressed to 690 fs. Taking into account the compression efficiency of 90% and the main peak energy proportion of 91%, the corresponding peak power is calculated to be 4 MW. The laser noise characteristics before and after CBC are examined, and the results indicate that the CBC would degrade the low frequency relative intensity noise (RIN), of which the integration is 1.74% in [100 Hz, 2 MHz] at the maximum combined output power. In addition, the effects of the nonlinear spectrum broadening during chirped pulse amplification on the CBC efficiency are also investigated, showing that a higher extent of pulse stretching is effective in alleviating the spectrum broadening and realizing a higher output power with decent combining efficiency.Graphical

Experimental demonstration of scene-based cophasing in optical synthetic aperture imaging using the SPGD algorithm

Large-aperture telescopes based on optical synthetic aperture imaging are investigated for recent high-resolution spaceborne observations. An enabling technique of aperture synthesis is a cophasing method to suppress a piston-tip-tilt error between sub-apertures. This paper proposes a scene-based cophasing technique using the stochastic parallel gradient descent (SPGD) algorithm, assuming application to high-resolution Earth observation. A significant advantage of the SPGD algorithm is a model-less cophasing capability based on extended scenes, but the simultaneous scene-based piston-tip-tilt correction between multiple apertures has not been demonstrated. In this paper, we developed a tabletop synthetic aperture imaging system with 37 sub-apertures and demonstrated extended-scene-based piston-tip-tilt control by optimizing applied voltages to 111 actuators simultaneously. The demonstration experiments used not only static scenes but also a time-varying dynamic scene for observation targets. In every measurement, the proposed scene-based approach reduced the initially defined piston-tip-tilt errors, and the image sharpness significantly improved, although the correction rate in the dynamic scene observation was slower. Finally, this paper discusses the influence of scene dynamics on image-based cophasing.

LSTM network optimization and task network construction based on heuristic algorithm

This work aims to advance the security management of complex networks to better align with evolving societal needs. The work employs the Ant Colony Optimization algorithm in conjunction with Long Short-Term Memory neural networks to reconstruct and optimize task networks derived from time series data. Additionally, a trend-based noise smoothing scheme is introduced to mitigate data noise effectively. The approach entails a thorough analysis of historical data, followed by applying trend-based noise smoothing, rendering the processed data more scientifically robust. Subsequently, the network reconstruction problem for time series data originating from one-dimensional dynamic equations is addressed using an algorithm based on the principles of Stochastic Gradient Descent (SGD). This algorithm decomposes time series data into smaller samples and yields optimal learning outcomes in conjunction with an adaptive learning rate SGD approach. Experimental results corroborate the remarkable fidelity of the weight matrix reconstructed by this algorithm to the true weight matrix. Moreover, the algorithm exhibits efficient convergence with increasing data volume, manifesting shorter time requirements per iteration while ensuring the attainment of optimal solutions. When the sample size remains constant, the algorithm’s execution time is directly proportional to the square of the number of nodes. Conversely, as the sample size scales, the SGD algorithm capitalizes on the availability of more information, resulting in improved learning outcomes. Notably, when the noise standard deviation is 0.01, models predicated on SGD and the Least-Squares Method (LSM) demonstrate reduced errors compared to instances with a noise standard deviation of 0.1, highlighting the sensitivity of LSM to noise. The proposed methodology offers valuable insights for advancing research in complex network studies.

Sidelobe suppression for accelerating beams

Although the control of trajectory, amplitude and beam-width in accelerating beams have been extensively investigated, sidelobes manipulation of such beams, which is required in many applications, has been surprisingly under-researched. This paper presents an approach for the generating of accelerating beams with significantly reduced sidelobes. The proposed method encompasses a two-step angular spectrum design, including employing a general model to establish the phase distribution and applying a stochastic parallel gradient descent (SPGD) algorithm to optimize the binary amplitude modulation. Experimental results confirm that the sidelobe intensity of accelerating beams can be reduced by over 50% with our method, thereby enhancing their applicability in many fields, such as micro-machining, particle manipulation, and optical communication.

Compensated DOE in a VHG-based waveguide display to improve uniformity

Augmented reality head-mounted displays (AR-HMDs) utilizing diffractive waveguides have emerged as a popular research focus. However, the illuminance uniformity over the fields of view (FOV) is often unsatisfactory in volume holographic grating (VHG) based waveguide displays. This paper proposes a high uniformity AR waveguide display system. Firstly, the angular uniformity of the VHG-based waveguide displays is analyzed. Subsequently, diffractive optical elements (DOEs) are seamlessly integrated onto the outer coupling surface of the waveguide substrate to improve the angular uniformity through phase compensation. To design the DOE phase, the multi-objective stochastic gradient descent (MO-SGD) algorithm is proposed. A single DOE is used to compensating various images form the image source. A hybrid loss, which includes the learned perceptual image patch similarity (LPIPS) metric, is applied to enhance the algorithm performance. Simulation results show that the proposed method effectively suppresses illumination degradation at the edge FOV in exit pupil images of the waveguide display system. In the results, the peak signal-to-noise ratio (PSNR) is improved by 5.54 dB. Optical experiments validate the effectiveness of the proposed method. The measured nonuniformity (NU) against FOVs is improved by 53.05% from 0.3749 to 0.1760.

Noninvasive holographic imaging through dynamically scattering media.

We present a noninvasive method for quantitative phase imaging through dynamically scattering media. A complex amplitude object, illuminated with coherent light, is captured through a dynamically scattering medium and a variable coded aperture, without the need for interferometric measurements or imaging optics. The complex amplitude of the object is computationally retrieved from intensity images that use multiple coded aperture patterns, employing a stochastic gradient descent algorithm. We demonstrate the proposed method both numerically and experimentally.

Task offloading scheme of vehicular cloud edge computing based on Digital Twin and improved A3C

In traditional vehicle edge computing research, the limited computing resources of edge servers are not fully considered. At the same time, when using deep reinforcement learning (DRL) to solve vehicle edge computing problems, it is not fully considered that DRL requires a large amount of real-time data and is prone to gradient explosion or disappearance, over-fitting and local optimal solution problems. Therefore, this article proposes a digital twin (DT)-assisted vehicular cloud edge computing offloading solution. To solve the problem of limited edge server computing resources, this article proposes a vehicle, cloud, and edge server collaboration solution. In response to the problem that DRL requires a large amount of real-time data, this paper proposes a real-time data acquisition method based on DT technology. Aiming at the gradient explosion or disappearance, overfitting, and local optimal solution problems of the asynchronous dominant actor–critic (A3C) algorithm, this paper proposes an improved algorithm of A3C, which introduces the ɛ-greedy strategy and dynamic baseline to increase the possibility of exploration and increase the generalization ability of the model through Dropout technology. During the training process, this article also uses the stochastic gradient descent (SGD) algorithm to accelerate the training process and reduce the computational complexity. The simulation results show that compared with other solutions, the improved algorithm proposed in this paper can effectively reduce the delay and energy consumption of offloading tasks at the vehicular edge cloud.

Generation of needle-shaped beam with super-resolution spot and large depth of focus using diffractive optics

The finite depth of focus (DOF) of a focusing lens gives rise to the degradation of lateral resolution at defocused positions. Consequently, the generation of a needle-shaped beam with sub-diffraction size and extended DOF can break through this limitation, but few studies have effectively balanced these two characteristics. In this paper, we propose a method based on the stochastic parallel gradient descent (SPGD) algorithm for designing multi-ring diffractive optical elements (DOEs) to produce the needle-shaped beam, which maintains a super-resolution and uniform spot within a large extended DOF range. Numerical simulations and experimental results have validated that our method can extend the DOF by 14 times longitudinally and reduce the lateral resolution to 73% of Airy spot, while maintaining up to 92% uniformity of the beam diameter. Furthermore, we demonstrate that the DOF of the needle-shaped beam extends with the increase of the number of rings and phase levels within the DOE. Our work provides an effective strategy for the design of DOF-extended DOE and it is anticipated to find potential applications in microscopy and three-dimensional imaging, photolithography, and optical manipulation.

Sentiment Analysis - An optimized Weighted Horizontal Ensemble approach

Sentiment Analysis has gained authority as one of the primary means of analyzing feedbacks and opinion by individuals, organizations and governments. The result of sentiment analysis informs an organization on areas to improve and how best to manage customers. While sentiment analysis may be misleading as no algorithm has been considered 100% efficient, the choice of algorithms can optimize the result based on the dataset in question. This paper aims at studying various algorithms and implementing a weighted horizontal ensemble algorithm as a panacea to low confidence level in the results of sentiment analysis. We designed a system that implements the original Naive Bayes algorithm, Multinomial Naïve Bayes algorithm, Bernoulli Native Bayes algorithm, Logistic Regression algorithm, Linear Support Vector Classifier algorithm and the Stochastic Gradient Descent algorithm. Our dataset was sourced from the Stanford University. It contains fifty thousand (50,000) movie reviews. Dataset from the Nigerian movie review was used to test the models. The reviews were encoded as a sequence of word indices. An accuracy of over 91% was achieved. The Ensemble technique delivered an F1-measure of 90%. Ensemble technique provides a more reliable confidence level on sentiment analysis. The researchers also discovered that change in writing style can affect the performance of sentiment analysis

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Systems and Computational Screening identifies SRC and NKIRAS2 as Baseline Correlates of Risk (CoR) for Live Attenuated Oral Typhoid Vaccine (TY21a) associated Protection

AimWe investigated the molecular underpinnings of variation in immune responses to the live attenuated typhoid vaccine (Ty21a) by analyzing the baseline immunological profile. We utilized gene expression datasets obtained from the Gene Expression Omnibus (GEO) database (accession number: GSE100665) before and after immunization. We then employed two distinct computational approaches to identify potential baseline biomarkers associated with responsiveness to the Ty21a vaccine. Main methodsThe first pipeline (knowledge-based) involved the retrieval of differentially expressed genes (DEGs), functional enrichment analysis, protein-protein interaction network construction, and topological network analysis of post-immunization datasets before gauging their pre-vaccination expression levels. The second pipeline utilized an unsupervised machine learning algorithm for data-driven feature selection on pre-immunization datasets. Supervised machine-learning classifiers were employed to computationally validate the identified biomarkers. Key findingsBaseline activation of NKIRAS2 (a negative regulator of NF-kB signalling) and SRC (an adaptor for immune receptor activation) was negatively associated with Ty21a vaccine responsiveness, whereas LOC100134365 exhibited a positive association. The Stochastic Gradient Descent (SGD) algorithm accurately distinguished vaccine responders and non-responders, with 88.8%, 70.3%, and 85.1% accuracy for the three identified genes, respectively. SignificanceThis dual-pronged novel analytical approach provides a comprehensive comparison between knowledge-based and data-driven methods for the prediction of baseline biomarkers associated with Ty21a vaccine responsiveness. The identified genes shed light on the intricate molecular mechanisms that influence vaccine efficacy from the host perspective while pushing the needle further towards the need for development of precise enteric vaccines and on the importance of pre-immunization screening.

A distributed learning based on robust diffusion SGD over adaptive networks with noisy output data

Outliers and noises are unavoidable factors that cause performance of the distributed learning algorithms to be severely reduced. Developing a robust algorithm is vital in applications such as system identification and forecasting stock market, in which noise on the desired signals may intensely divert the solutions. In this paper, we propose a Robust Diffusion Stochastic Gradient Descent (RDSGD) algorithm based on the pseudo-Huber loss function which can significantly suppress the effect of Gaussian and non-Gaussian noises on estimation performances in the adaptive networks. Performance and convergence behavior of RDSGD are assessed in presence of the α-stable and Mixed-Gaussian noises in the stationary and non-stationary environments. Simulation results show that the proposed algorithm can achieve both higher convergence rate and lower steady-state misadjustment than the conventional diffusion algorithms and several robust algorithms.

Differentially private stochastic gradient descent with low-noise

Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance of developing privacy-preserving machine learning algorithms that ensure good performance while preserving privacy. In this paper, we focus on the privacy and utility (measured by excess risk bounds) performances of differentially private stochastic gradient descent (SGD) algorithms in the setting of stochastic convex optimization. Specifically, we examine the pointwise problem in the low-noise setting for which we derive sharper excess risk bounds for the differentially private SGD algorithm. In the pairwise learning setting, we propose a simple differentially private SGD algorithm based on gradient perturbation. Furthermore, we develop novel utility bounds for the proposed algorithm, proving that it achieves optimal excess risk rates even for non-smooth losses. Notably, we establish fast learning rates for privacy-preserving pairwise learning under the low-noise condition, which is the first of its kind.

On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design

Bayesian Experimental Design (BED), which aims to find the optimal experimental conditions for Bayesian inference, is usually posed as to optimize the expected information gain (EIG). The gradient information is often needed for efficient EIG optimization, and as a result the ability to estimate the gradient of EIG is essential for BED problems. The primary goal of this work is to develop methods for estimating the gradient of EIG, which, combined with the stochastic gradient descent algorithms, result in efficient optimization of EIG. Specifically, we first introduce a posterior expected representation of the EIG gradient with respect to the design variables. Based on this, we propose two methods for estimating the EIG gradient, UEEG-MCMC that leverages posterior samples generated through Markov Chain Monte Carlo (MCMC) to estimate the EIG gradient, and BEEG-AP that focuses on achieving high simulation efficiency by repeatedly using parameter samples. Theoretical analysis and numerical studies illustrate that UEEG-MCMC is robust agains the actual EIG value, while BEEG-AP is more efficient when the EIG value to be optimized is small. Moreover, both methods show superior performance compared to several popular benchmarks in our numerical experiments.

Deep learning enhanced drug discovery for novel biomaterials in regenerative medicine utilizing graph neural network approach for predicting cellular responses

This study introduces a novel approach to drug discovery in regenerative medicine through the utilization of a graph neural network (GNN). The research methodology integrates the development and training of the GNN with a subsequent evaluation of its performance metrics. The first phase involves the generation of synthetic data simulating a biological network, employing networkX and NumPy libraries to construct a random graph with Erdos-Renyi topology. The data, representing cellular responses to biomaterials, is then converted into PyTorch tensors for compatibility with the GNN architecture. The GNN model, characterized by two fully connected layers with ReLU and log-softmax activations, captures intricate relationships within the graph-structured data. The second phase employs a stochastic gradient descent algorithm, specifically the Adam optimizer, to train the GNN over 100 epochs using the cross-entropy loss for multi-class classification. The research methodology extends to the evaluation phase, producing three distinct output graphs for analysis: Visualization of the graph structure, a comparison between predicted and true labels, and a plot illustrating training loss over epochs. Performance metrics, including accuracy, precision, recall, and F1-score, are computed to assess the model’s predictive capabilities quantitatively. The study concludes with a discussion on the nuances revealed by each graph and their implications for refining GNN models in the context of drug discovery for regenerative medicine.

Photonic integrated coherent beam combiner based on MMI and the SPGD algorithm.

Atmospheric turbulence severely degrades the optical wavefront of a propagating beam, which greatly reduces the coupling efficiency of free-space optical (FSO) receivers. Among the various methods to mitigate the effects, the use of a multi-channel receiver is more convenient and economical. After passing through the multi-channel receiver, multiple single-mode fibers (SMFs) are output and need to be combined. In this paper, we propose photonic integrated coherent beam combiners based on multimode interference (MMI) and the stochastic parallel gradient descent (SPGD) algorithm, which avoids detecting the light out of each channel and adding the data signal in the electrical domain. First, we propose a 4-channel coherent beam combiner based on a 4×1 MM, and about 21 iterations of the SPGD algorithm are required to enhance the combined optical power to a maximum of 96%. Furthermore, we demonstrate a combination of 16 beams using five 4×1 MMIs, which requires 140 iterations to enhance the combined power to 89%. This study offers theoretical insights to enhance the integration of FSO communication systems.

Deep Reinforcement Learning With Reward Design for Quantum Control

Deep reinforcement learning (DRL) has been recognized as a powerful tool in quantum physics, where DRL's reward design is non-trivial but crucial for quantum control tasks. To address the problem of over-reliance on human empirical knowledge to design DRL's rewards, we propose a deep reinforcement learning approach with a novel reward paradigm designed by the learning process information (DRL-LPI), where the learning process information (LPI) comprises the state information and the experiences. In DRL-LPI, the state information after being classified by a fidelity threshold, and the experiences are first stored simultaneously in the respective sequences, and this process is repeated until a similar-segment ends. Then, the stored state information is converted to the real value and used to design the reward value by applying a self-amplitude function. Next, the designed reward values are integrated with the stored experiences to compose transitions for DRL's training. Through comparisons to five representative reward schemes, the proposed DRL-LPI is evaluated on two typical quantum control tasks, i.e., the spin-(1/2) quantum state control and many-coupled qubits state control, and the experimental results show the superior learning efficiency and control performance of the proposed approach. More results show that DRL-LPI exhibits the ability to learn the control strategy with few control actions compared to stochastic gradient descent (SGD) and genetic algorithm (GA).

Polynomial convergence of iterations of certain random operators in Hilbert space

Abstract We study the convergence of a random iterative sequence of a family of operators on infinite-dimensional Hilbert spaces, inspired by the stochastic gradient descent (SGD) algorithm in the case of the noiseless regression. We identify conditions that are strictly broader than previously known for polynomial convergence rate in various norms, and characterize the roles the randomness plays in determining the best multiplicative constants. Additionally, we prove almost sure convergence of the sequence.

CDH1 gene as biomarker towards breast cancer prediction*

Breast cancer is considered to be happened due to genetic aberration. Out of several genes expressed, it is found that cadherin 1, type 1 (CDH1) is responsible in several ways to control the metabolic order in human. Deregulation of the function of protein E-cadherin, expressed from CDH1 plays an important role in lobular breast cancer. In order to understand the root cause of this recent claim, we focus on CDH1 gene: whether the genetic information translated due to any deviation/alteration/modification in its sequence is related to the occurrence of the different types breast cancer. Towards this end, quantitative analysis of different biophysical and bio-chemical properties of CDH1 gene in genomic and proteomic levels from the available genomic (cDNA) sequences of CDH1 gene (obtained from the COSMIC Database for 78 patients, suffering from various types of breast cancer) clearly emphasizes that alternation/modification in the sequence of the CDH1 gene can be detrimental. Furthermore, Random forest, K-nearest neighbour and stochastic gradient descent (SGD) algorithms are applied on the derived dataset to classify the types of breast cancer, and to validate our hypothesis regarding the acute role of CDH1 as potential bio marker for breast cancer. Analysis of the mutated CDH1 gene sequences, and their related parameters using aforesaid machine learning techniques clearly establish that CDH1 gene can take the deterministic role in predicting the chances of occurrences of different types of breast cancer with an accuracy of > 90 % . Such an observation opens a new paradigm in diagnostic approach of breast cancer. Communicated by Ramaswamy H. Sarma

Breast Cancer Prediction Based on Multiple Machine Learning Algorithms.

The incidence of breast cancer has steadily risen over the years owing to changes in lifestyle and environment. Presently, breast cancer is one of the primary causes of cancer-related deaths among women, making it a crucial global public health concern. Thus, the creation of an automated diagnostic system for breast cancer bears great importance in the medical community. This study analyses the Wisconsin breast cancer dataset and develops a machine learning algorithm for accurately classifying breast cancer as benign or malignant. Our research is a retrospective study, and the main purpose is to develop a high-precision classification algorithm for benign and malignant breast cancer. To achieve this, we first preprocessed the dataset using standard techniques such as feature scaling and handling missing values. We assessed the normality of the data distribution initially, after which we opted for Spearman correlation analysis to examine the relationship between the feature subset data and the labeled data, considering the normality test results. We subsequently employed the Wilcoxon rank sum test to investigate the dissimilarities in distribution among various breast cancer feature data. We constructed the feature subset based on statistical results and trained 7 machine learning algorithms, specifically the decision tree, stochastic gradient descent algorithm, random forest algorithm, support vector machine algorithm, logistics algorithm, and AdaBoost algorithm. The results of the evaluation indicated that the AdaBoost-Logistic algorithm achieved an accuracy of 99.12%, outperforming the other 6 algorithms and previous techniques. The constructed AdaBoost-Logistic algorithm exhibits significant precision with the Wisconsin breast cancer dataset, achieving commendable classification performance for both benign and malignant breast cancer cases.