Network Sparsity Research Articles

Selective Gradient Dropout in Deep Neural Networks for Linear B cell Epitope Prediction

This study focused on determining linear B-cell epitopes using a dataset of 50,000 experimentally validated linear epitopes from IEDB. The analysis showed that sequence-derived features were crucial for predicting BCEs. A deep learning framework was developed to improve feature representation and efficiency. Despite the power of deep neural networks, they require regularization technique to perform well. However, applying dropout to crucial recurrent connections in networks like LSTM can disrupt information flow. To address this, we introduce a novel method based on selective gradient dropout for deep fully connected layers. This novel method selectively freezes specific connections during training, improving network sparsity and efficiency while maintaining performance. Experimental results confirm its effectiveness in mitigating overfitting and enhancing computational efficiency. Experimental results show that this sparse network outperforms traditional methods in classification tasks achieving an average 86.3% accuracy. Thus being a significant improvement compared to existing tools.

MQGA: A quantitative analysis of brain network hubs using multi-graph theoretical indices

Recent advancements in large-scale network studies have shown that connector hubs and provincial hubs are vital for coordinating complex cognitive tasks by facilitating information transfer between and within specialized modules. However, current methods for identifying these hubs often lack standardized measurement criteria, hindering quantitative analysis. This study proposes a novel computational method utilizing multi-graph theoretical index calculations to quantitatively analyze hub attributes in brain networks. Using benchmark network, random simulation network (N = 100), resting fMRI data from the ADHD-200 NYU dataset (HC = 110, ADHD = 146), and the Peking dataset (HC = 120, ADHD = 83), we introduce the Multi-criteria Quantitative Graph Analysis (MQGA) method, which employs betweenness centrality, degree centrality, and participation coefficient to determine the connector (con) hub index and provincial (pro) hub index. The method's accuracy, reliability, and stability were validated through correlation analysis of hub indices and labels, vulnerability tests, and consistency analysis across subjects. Results indicate that as network sparsity increases, the con hub index increases while the pro hub index decreases, with the optimal hub node index at 4 % sparsity. Vulnerability tests revealed that removing con nodes had a greater impact on network integrity than removing pro nodes. Both con and pro exhibited stability in consistency analyses, but con was more stable. The stability of hub scores in disease groups was significantly lower than in the healthy control group. High con values were found in the precuneus, postcentral gyrus, and precentral gyrus, whereas high pro values were identified in the precentral gyrus, postcentral gyrus, superior parietal lobule, precuneus, and superior temporal gyrus. This approach enhances the accuracy and sensitivity of hub node identification, facilitating precise comparisons and producing consistent, replicable results, advancing our understanding of brain network hub nodes, their roles in cognitive processes, and their implications for brain disease research.

Topological conditions drive stability in meta-ecosystems

On a global level, ecological communities are being perturbed at an unprecedented rate by human activities and environmental instabilities. Yet, we understand little about what factors facilitate or impede long-term persistence of these communities. While observational studies indicate that increased biodiversity must, somehow, be driving stability, theoretical studies have argued the exact opposite viewpoint instead. This encouraged many researchers to participate in the ongoing debate. Within this context, however, there has been a severe lack of studies that consider spatial features explicitly, even though nearly all habitats are spatially embedded. To this end, we study here the linear stability of meta-ecosystems on networks that describe how discrete patches are connected by dispersal between them. By combining results from random-matrix theory and network theory, we are able to show that there are three distinct features that underlie stability: edge density, tendency to triadic closure, and isolation or fragmentation. Our results appear to further indicate that network sparsity does not necessarily reduce stability, and that connections between patches are just as important, if not more so, to consider when studying the stability of large ecological systems. Published by the American Physical Society 2024

Evaluating the Impact of Interferogram Networks on the Performance of Phase Linking Methods

In recent years, phase linking (PL) methods in radar time-series interferometry (TSI) have proven to be powerful tools in geodesy and remote sensing, enabling the precise monitoring of surface displacement and deformation. While these methods are typically designed to operate on a complete network of interferograms, generating such networks is often challenging in practice. For instance, in non-urban or vegetated regions, decorrelation effects lead to significant noise in long-term interferograms, which can degrade the time-series results if included. Additionally, practical issues such as gaps in satellite data, poor acquisitions, or systematic errors during interferogram generation can result in incomplete networks. Furthermore, pre-existing interferogram networks, such as those provided by systems like COMET-LiCSAR, often prioritize short temporal baselines due to the vast volume of data generated by satellites like Sentinel-1. As a result, complete interferogram networks may not always be available. Given these challenges, it is critical to understand the applicability of PL methods on these incomplete networks. This study evaluated the performance of two PL methods, eigenvalue decomposition (EVD) and eigendecomposition-based maximum-likelihood estimator of interferometric phase (EMI), under various network configurations including short temporal baselines, randomly sparsified networks, and networks where low-coherence interferograms have been removed. Using two sets of simulated data, the impact of different network structures on the accuracy and quality of the results was assessed. These patterns were then applied to real data for further comparison and analysis. The findings demonstrate that while both methods can be effectively used on short temporal baselines, their performance is highly sensitive to network sparsity and the noise introduced by low-coherence interferograms, requiring careful parameter tuning to achieve optimal results across different study areas.

Structural and covariance network alterations of the hippocampus and amygdala in congenital hearing loss children

ObjectiveThe hippocampus and amygdala, as important components of the limbic system, play crucial roles in central remodeling in congenital hearing loss. This study aimed to investigate the morphological integrity and network properties of the subfields of hippocampus and amygdala in children with congenital hearing loss. MethodsA total of 24 children with congenital hearing loss and 17 age- and sex- matched healthy controls (HC) are included in the study. T1-weighted images are analyzed by segmenting the brain into cortical and subcortical regions. Intergroup difference of volumes were explored. Structural covariance networks for the whole brain and hippocampus-amygdala subregions were constructed. Between-group differences of network property are investigated by comparing area under a range of network sparsity. ResultsPatients with congenital hearing loss exhibited significantly larger volumes in the right dentate gyrus and CA3 of the hippocampus. However, there were no significant differences in total hippocampal or showed decreased global efficiency and increased characteristic path length, indicating reduced network integration. Lower betweenness centrality was observed in the left hippocampal fissure in the hearing loss group. The changes in volume and network topological properties are not affected by age and sex. ConclusionChildren with congenital hearing loss display specific volumetric increases in hippocampal subregions, suggesting compensatory adaptations to auditory deprivation. The hippocampus-amygdala network shows significant reorganization, potentially underpinning cognitive and behavioral development issues associated with congenital hearing loss. These findings highlight the importance of targeted neural substrates in understanding and addressing the developmental challenges faced by children with congenital hearing loss.

Pruning convolutional neural networks for inductive conformal prediction

Neural network pruning is a popular approach to reducing model storage size and inference time by removing redundant parameters in the neural network. However, the uncertainty of predictions from pruned models is unexplored. In this paper we study neural network pruning in the context of conformal predictors (CP). The conformal prediction framework built on top of machine learning algorithms supplements their predictions with reliable uncertainty measure in the form of prediction sets, under the independent and identically distributed assumption on the data. Convolutional neural networks (CNNs) have complicated architectures and are widely used in various applications nowadays. Therefore, we focus on pruning CNNs and, in particular, filter-level pruning. We first propose a brute force method that estimates the contribution of a filter to the CP’s predictive efficiency and removes those with the least contribution. Given the computation inefficiency of the brute force method, we also propose the Taylor expansion to approximate the filter’s contribution. Furthermore, we improve the global pruning method by protecting the most important filters within each layer from being pruned. In addition, we explore the ConfTr loss function which is optimized to yield maximal CP efficiency in the context of neural network pruning. We have conducted extensive experimental studies and compared the results regarding the trade-offs between predictive efficiency, computational efficiency, and network sparsity. These results are instructive for deploying pruned neural networks with applications using conformal prediction where reliable predictions and reduced computational cost are relevant, such as in safety-critical applications.

A detection method of oil content for maize kernels based on CARS feature selection and deep sparse autoencoder feature extraction

To achieve accurate, rapid, and non-destructive oil content determination in maize, a neural fitting model that combines feature selection and feature extraction is proposed. Competitive adaptive re-weighted sampling (CARS) is used to choose features, while the deep sparse autoencoder (DSAE) network is utilized to extract features. The 160 spectral data from the near-infrared spectra public datasets of maize served as experimental samples. Several preprocessing approaches were tested, and the first-order derivatives fared the best. To capture the various degrees of abstract features in spectral data, SSAE networks with two or three hidden layers are developed. The number of nodes in the network's hidden layer (30, 50, 70, 100, 200, 300, respectively) and the network's sparsity parameter (0.001, 0.004, 0.007, 0.01, 0.04, 0.07, 0.1, 0.2, …, 1) are tuned to generate partial least squares regression (PLSR) models for testing the feature learning effect. The highest performance comes from a three-layer network with 200 hidden layer nodes and a sparsity parameter of 0.04. This network extracts three levels of features from the preprocessed spectral data: F1, F2, and F3. Full_cars, F1_cars, F2_cars, and F3_cars are obtained using the CARS algorithm for variable selection on the preprocessed spectral data, F1, F2, and F3, respectively. They are then integrated into a final feature set to build a neural-fitting network (FNN). After training and testing, the test set has a correlation coefficient of 0.964205 and an mean square error (MSE) of 0.00636809. Comparing this model to support vector machine regression (SVR), PLSR, and gaussian process regression (GPR), the findings demonstrate that the model described in this study outperforms the others in terms of fitting accuracy and generalization performance. To summarize, this paper introduces a new regression analysis method based on near-infrared spectra.

Complex-valued soft-log threshold reweighting for sparsity of complex-valued convolutional neural networks

Complex-valued convolutional neural networks (CVCNNs) have been demonstrated effectiveness in classifying complex signals and synthetic aperture radar (SAR) images. However, due to the introduction of complex-valued parameters, CVCNNs tend to become redundant with heavy floating-point operations. Model sparsity is emerged as an efficient method of removing the redundancy without much loss of performance. Currently, there are few studies on the sparsity problem of CVCNNs. Therefore, a complex-valued soft-log threshold reweighting (CV-SLTR) algorithm is proposed for the design of sparse CVCNN to reduce the number of weight parameters and simplify the structure of CVCNN. On one hand, considering the difference between complex and real numbers, we redefine and derive the complex-valued log-sum threshold method. On the other hand, by considering the distinctive characteristics of complex-valued convolutional (CConv) layers and complex-valued fully connected (CFC) layers of CVCNNs, the complex-valued soft and log-sum threshold methods are respectively developed to prune the weights of different layers during the forward propagation, and the sparsity thresholds are optimized during the backward propagation by inducing a sparsity budget. Furthermore, different optimizers can be integrated with CV-SLTR. When stochastic gradient descent (SGD) is used, the convergence of CV-SLTR is proved if Lipschitzian continuity is satisfied. Experiments on the RadioML 2016.10A and S1SLC-CVDL datasets show that the proposed algorithm is efficient for the sparsity of CVCNNs. It is worth noting that the proposed algorithm has fast sparsity speed while maintaining high classification accuracy. These demonstrate the feasibility and potential of the CV-SLTR algorithm.

Accurate and Efficient Event-Based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network.

Spiking neural networks (SNNs), known for their low-power, event-driven computation, and intrinsic temporal dynamics, are emerging as promising solutions for processing dynamic, asynchronous signals from event-based sensors. Despite their potential, SNNs face challenges in training and architectural design, resulting in limited performance in challenging event-based dense prediction tasks compared with artificial neural networks (ANNs). In this work, we develop an efficient spiking encoder-decoder network (SpikingEDN) for large-scale event-based semantic segmentation (EbSS) tasks. To enhance the learning efficiency from dynamic event streams, we harness the adaptive threshold which improves network accuracy, sparsity, and robustness in streaming inference. Moreover, we develop a dual-path spiking spatially adaptive modulation (SSAM) module, which is specifically tailored to enhance the representation of sparse events and multimodal inputs, thereby considerably improving network performance. Our SpikingEDN attains a mean intersection over union (MIoU) of 72.57% on the DDD17 dataset and 58.32% on the larger DSEC-Semantic dataset, showing competitive results to the state-of-the-art ANNs while requiring substantially fewer computational resources. Our results shed light on the untapped potential of SNNs in event-based vision applications. The source codes are publicly available at https://github.com/EMI-Group/spikingedn.

Generalized sparse and outlier-robust broad learning systems for multi-dimensional output problems

Broad learning systems (BLSs) are becoming increasingly popular due to their fast and superior learning capabilities. However, their performances are susceptible to outliers and dense networks. The Elastic-net robust BLS (ER-BLS), which employs an ℓ1-norm loss function along with Elastic-net regularization, attempts to mitigate these issues by promoting network sparsity. Nevertheless, even ER-BLS fails to achieve satisfactory robustness and sparsity in multi-dimensional output tasks because the ℓ1-norm ignores the correlations between the output dimensions and output weights. Therefore, we propose a generalized Elastic-net robust BLS (GER-BLS) by generalizing the ℓ1-norm to the L2,1-norm. The L2,1-norm enhances robustness by treating the residuals across all output dimensions as a whole and induces node-level sparsity by penalizing all output weights connected to a single hidden node. The alternating direction method of multipliers (ADMM) algorithm was employed to solve GER-BLS efficiently, circumventing the non-differentiable issue of the objective function. Moreover, we developed DGER-BLS, a distributed variant of GER-BLS, tailored to handle data with distributed storage. The convergence analysis of our proposed algorithms and a large number of numerical experiments on multi-dimensional output datasets demonstrated the effectiveness of GER-BLS and DGER-BLS in dealing with outliers, distributed stored data, and sparsity.

CVGAE: A Self-Supervised Generative Method for Gene Regulatory Network Inference Using Single-Cell RNA Sequencing Data.

Gene regulatory network (GRN) inference based on single-cell RNA sequencing data (scRNAseq) plays a crucial role in understanding the regulatory mechanisms between genes. Various computational methods have been employed for GRN inference, but their performance in terms of network accuracy and model generalization is not satisfactory, and their poor performance is caused by high-dimensional data and network sparsity. In this paper, we propose a self-supervised method for gene regulatory network inference using single-cell RNA sequencing data (CVGAE). CVGAE uses graph neural network for inductive representation learning, which merges gene expression data and observed topology into a low-dimensional vector space. The well-trained vectors will be used to calculate mathematical distance of each gene, and further predict interactions between genes. In overall framework, FastICA is implemented to relief computational complexity caused by high dimensional data, and CVGAE adopts multi-stacked GraphSAGE layers as an encoder and an improved decoder to overcome network sparsity. CVGAE is evaluated on several single cell datasets containing four related ground-truth networks, and the result shows that CVGAE achieve better performance than comparative methods. To validate learning and generalization capabilities, CVGAE is applied in few-shot environment by change the ratio of train set and test set. In condition of few-shot, CVGAE obtains comparable or superior performance.

Static graph approximations of dynamic contact networks for epidemic forecasting.

Epidemic modeling is essential in understanding the spread of infectious diseases like COVID-19 and devising effective intervention strategies to control them. Recently, network-based disease models have integrated traditional compartment-based modeling with real-world contact graphs and shown promising results. However, in an ongoing epidemic, future contact network patterns are not observed yet. To address this, we use aggregated static networks to approximate future contacts for disease modeling. The standard method in the literature concatenates all edges from a dynamic graph into one collapsed graph, called the full static graph. However, the full static graph often leads to severe overestimation of key epidemic characteristics. Therefore, we propose two novel static network approximation methods, DegMST and EdgeMST, designed to preserve the sparsity of real world contact network while remaining connected. DegMST and EdgeMST use the frequency of temporal edges and the node degrees respectively to preserve sparsity. Our analysis show that our models more closely resemble the network characteristics of the dynamic graph compared to the full static ones. Moreover, our analysis on seven real-world contact networks suggests EdgeMST yield more accurate estimations of disease dynamics for epidemic forecasting when compared to the standard full static method.

Assessing the potential impact of assimilating total surface current velocities in the Met Office’s global ocean forecasting system

Accurate prediction of ocean surface currents is important for marine safety, ship routing, tracking of pollutants and in coupled forecasting. Presently, velocity observations are not routinely assimilated in global ocean forecasting systems, largely due to the sparsity of the observation network. Several satellite missions are now being proposed with the capability to measure Total Surface Current Velocities (TSCV). If successful, these would substantially increase the coverage of ocean current observations and could improve accuracy of ocean current forecasts through data assimilation. In this paper, Observing System Simulation Experiments (OSSEs) are used to assess the impact of assimilating TSCV in the Met Office’s global ocean forecasting system. Synthetic observations are generated from a high-resolution model run for all standard observation types (sea surface temperature, profiles of temperature and salinity, sea level anomaly and sea ice concentration) as well as TSCV observations from a Sea surface KInematics Multiscale monitoring (SKIM) like satellite. The assimilation of SKIM like TSCV observations is tested over an 11 month period. Preliminary experiments assimilating idealised single TSCV observations demonstrate that ageostrophic velocity corrections are not well retained in the model. We propose a method for improving ageostrophic currents through TSCV assimilation by initialising Near Inertial Oscillations with a rotated incremental analysis update (IAU) scheme. The OSSEs show that TSCV assimilation has the potential to significantly improve the prediction of velocities, particularly in the Western Boundary Currents, Antarctic Circumpolar Current and in the near surface equatorial currents. For global surface velocity the analysis root-mean-square-errors (RMSEs) are reduced by 23% and there is a 4-day gain in forecast RMSE. There are some degradations to the subsurface in the tropics, generally in regions with complex vertical salinity structures. However, outside of the tropics, improvements are seen to velocities throughout the water column. Globally there are also improvements to temperature and sea surface height when TSCV are assimilated. The TSCV assimilation largely corrects the geostrophic ocean currents, but results using the rotated IAU method show that the energy at inertial frequencies can be improved with this method. Overall, the experiments demonstrate significant potential benefit of assimilating TSCV observations in a global ocean forecasting system.

Associations between satellite-derived estimates of PM2.5 species concentrations for organic carbon, elemental carbon, nitrate, and sulfate with birth weight and preterm birth in California during 2005-2014.

Characterizing the spatial distribution of PM2.5 species concentrations is challenging due to the geographic sparsity of the stationary monitoring network. Recent advances have enabled valid estimation of PM2.5 species concentrations using satellite remote sensing data for use in epidemiologic studies. In this study, we used satellite-based estimates of ambient PM2.5 species concentrations to estimate associations with birth weight and preterm birth in California. Daily 24 h averaged ground-level PM2.5 species concentrations of organic carbon, elemental carbon, nitrate, and sulfate were estimated during 2005-2014 in California at 1 km resolution. Birth records were linked to ambient pollutant exposures based on maternal residential zip code. Linear regression and Cox regression were conducted to estimate the effect of 1 µg/m3 increases in PM2.5 species concentrations on birth weight and preterm birth. Analyses included 4.7 million live singleton births having a median 28 days with exposure measurements per pregnancy. In single pollutant models, the observed changes in mean birth weight (per 1 µg/m3 increase in speciated PM2.5 concentrations) were: organic carbon -3.12 g (CI: -4.71, -1.52), elemental carbon -14.20 g (CI: -18.76, -9.63), nitrate -5.51 g (CI: -6.79, -4.23), and sulfate 9.26 g (CI: 7.03, 11.49). Results from multipollutant models were less precise due to high correlation between pollutants. Associations with preterm birth were null, save for a negative association between sulfate and preterm birth (Hazard Ratio per 1 µg/m3 increase: 0.973 CI: 0.958, 0.987).

Brain topology improved spiking neural network for efficient reinforcement learning of continuous control.

The brain topology highly reflects the complex cognitive functions of the biological brain after million-years of evolution. Learning from these biological topologies is a smarter and easier way to achieve brain-like intelligence with features of efficiency, robustness, and flexibility. Here we proposed a brain topology-improved spiking neural network (BT-SNN) for efficient reinforcement learning. First, hundreds of biological topologies are generated and selected as subsets of the Allen mouse brain topology with the help of the Tanimoto hierarchical clustering algorithm, which has been widely used in analyzing key features of the brain connectome. Second, a few biological constraints are used to filter out three key topology candidates, including but not limited to the proportion of node functions (e.g., sensation, memory, and motor types) and network sparsity. Third, the network topology is integrated with the hybrid numerical solver-improved leaky-integrated and fire neurons. Fourth, the algorithm is then tuned with an evolutionary algorithm named adaptive random search instead of backpropagation to guide synaptic modifications without affecting raw key features of the topology. Fifth, under the test of four animal-survival-like RL tasks (i.e., dynamic controlling in Mujoco), the BT-SNN can achieve higher scores than not only counterpart SNN using random topology but also some classical ANNs (i.e., long-short-term memory and multi-layer perception). This result indicates that the research effort of incorporating biological topology and evolutionary learning rules has much in store for the future.

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Neural network sparsity has attracted many research interests due to its similarity to biological schemes and high energy efficiency. However, existing methods depend on long-time training or fine-tuning, which prevents large-scale applications. Recently, some works focusing on post-training sparsity (PTS) have emerged. They get rid of the high training cost but usually suffer from distinct accuracy degradation due to neglect of the reasonable sparsity rate at each layer. Previous methods for finding sparsity rates mainly focus on the training-aware scenario, which usually fails to converge stably under the PTS setting with limited data and much less training cost. In this paper, we propose a fast and controllable post-training sparsity (FCPTS) framework. By incorporating a differentiable bridge function and a controllable optimization objective, our method allows for rapid and accurate sparsity allocation learning in minutes, with the added assurance of convergence to a predetermined global sparsity rate. Equipped with these techniques, we can surpass the state-of-the-art methods by a large margin, e.g., over 30\% improvement for ResNet-50 on ImageNet under the sparsity rate of 80\%. Our plug-and-play code and supplementary materials are open-sourced at https://github.com/ModelTC/FCPTS.

Tourist Flows and Multi-Destination Trips: A Network Analysis

ABSTRACT Conventional approaches to modeling multi-destination trips are either based on trade-offs between one single destination and multiple destinations or on a sequence of destinations in a linear manner. These approaches are inadequate to capture the manifestation of multi-destination trips as a network at the aggregate level. Hence, built upon network analytical frameworks, this study adopted a directed network to model multi-destination trips. Using a visitor profile survey of Hong Kong, we found that the networks of multi-destination trips for both leisure and business travelers have several interesting properties, such as high densities, high clustering coefficients, and short diameters. High-degree nodes represent either destinations or origins in the leisure network, but they have a dual role as both destinations and origins in the business network. Network sparsity has a significant impact on the topology of the leisure network, but the impact is negligible on the business network.

A strength and sparsity preserving algorithm for generating weighted, directed networks with predetermined assortativity

Degree-preserving rewiring is a widely used technique for generating unweighted networks with given assortativity, but for weighted networks, it is unclear how an analog would preserve the strengths and other critical network features such as sparsity level. This study introduces a novel approach for rewiring weighted networks to achieve desired directed assortativity. The method utilizes a mixed integer programming framework to establish a target network with predetermined assortativity coefficients, followed by an efficient rewiring algorithm termed “strength and sparsity preserving rewiring” (SSPR). SSPR retains the node strength distributions and network sparsity after rewiring. It is also possible to accommodate additional properties like edge weight distribution, albeit with extra computational cost. The optimization scheme can be used to determine feasible assortativity ranges for an initial network. The effectiveness of the proposed SSPR algorithm is demonstrated through its application to two classes of popular network models.

Biobjective gradient descent for feature selection on high dimension, low sample size data.

Even though deep learning shows impressive results in several applications, its use on problems with High Dimensions and Low Sample Size, such as diagnosing rare diseases, leads to overfitting. One solution often proposed is feature selection. In deep learning, along with feature selection, network sparsification is also used to improve the results when dealing with high dimensions low sample size data. However, most of the time, they are tackled as separate problems. This paper proposes a new approach that integrates feature selection, based on sparsification, into the training process of a deep neural network. This approach uses a constrained biobjective gradient descent method. It provides a set of Pareto optimal neural networks that make a trade-off between network sparsity and model accuracy. Results on both artificial and real datasets show that using a constrained biobjective gradient descent increases the network sparsity without degrading the classification performances. With the proposed approach, on an artificial dataset, the feature selection score reached 0.97 with a sparsity score of 0.92 with an accuracy of 0.9. For the same accuracy, none of the other methods reached a feature score above 0.20 and sparsity score of 0.35. Finally, statistical tests validate the results obtained on all datasets.

Neural network informed day-ahead scheduling of pumped hydro energy storage

This paper presents a neural network-constrained optimization model for the optimal scheduling of pumped hydro energy storage. Neural networks are trained offline to capture the complex head-dependent performance curves in both pump and turbine modes using actual operation data. The trained models are then embedded into the optimization framework that yields the optimal and physics-compliant day-ahead scheduling in energy and reserve markets for the pumped hydro energy storage. To identify the trade-off between modeling accuracy and computation burden, different neural network architectures are investigated, along with the impact of neural network sparsity, i.e., weights pruning to reduce dimensionality. The proposed approach is then compared with state-of-the-art solutions, such as piecewise linear approximations. To that end, a detailed simulator of the pumped hydro energy storage, mimicking its minute-wise behavior, is developed to accurately assess the feasibility and economic performance of the resulting schedules. Results demonstrate the ability of neural networks to better guide the optimization model, thus leading to higher profits while keeping acceptable solving times, especially when weight pruning is leveraged. In particular, we show that accurately capturing the non-linear characteristics of pumped hydro energy storage is critical to offer reliable reserve commitments to power systems.