Quality Of Transmission Estimation Research Articles

On the generalization of cognitive optical networking applications using composable machine learning

Model generalization characterizes the sustainability of machine learning (ML) designs applied to novel system states and therefore plays a vital role toward the realization of cognitive networking. In this paper, we present a composable ML framework (namely, CompML), aiming at generalizing ML-aided cognitive applications for optical networks. CompML makes use of three basic functional modules, i.e., the Loading, Recursion, and Readout modules, to model the loading/initialization processes (e.g., the launch of a signal), extract cumulative features by recursive operations, and produce model inferences, respectively. By the composition of the three modules and adoption of an end-to-end training mechanism, CompML allows for generalizing multiple tasks of the same domain [e.g., quality-of-transmission (QoT) estimation for different lightpaths]. We perform case studies of CompML on QoT estimation and nonlinearity compensation using both simulation and experimental data. Results show the superior generalization ability of CompML compared with the baselines, achieving mean absolute error (MAE) for generalized signal-to-noise ratio (GSNR) prediction error of below 1.06 dB for unseen lightpaths and up to 3 dB Q-factor improvement for nonlinearity compensation.

Local and global optimization methods for optical line control based on quality of transmission

The ever-increasing demand for data traffic in recent decades has pushed network operators to give importance to the aspect of infrastructure control to facilitate its scalability and maximize its capacity. A generic lightpath (LP) is deployed starting from a traffic request between a given pair of nodes in a network. LPs are operated in the network based on an estimate of the quality of transmission (QoT), which is derived from the physical layer characteristics of a selected route. Regardless of the model used to estimate QoT, it is necessary to calibrate the model to maximize its accuracy and define minimum design margins. The model calibration process depends significantly on the type of data that can be collected in the field (i.e., type of metric, resolution) and therefore on the available monitoring devices. In this work, a systematic evaluation of the QoT estimation is carried out on a multi-span erbium-doped-fiber-amplified optical line system (OLS) using in the first case only total power monitors and in the second experimentally emulating optical channel monitors (OCMs). Given the type of monitoring devices available, three different physical models are calibrated, and six optimization methods are used to define the optimal configuration of the target gain and tilt parameters of the optical amplifiers, jointly optimizing the working point of all amplifiers (global approach) or proceeding span by span (local approach). Subsequently, the OLS was set in each configuration obtained, and the generalized signal-to-noise ratio (GSNR) profile was measured at the end.

Evolutionary neuron-level transfer learning for QoT estimation in optical networks

To ensure the reliability and efficiency of lightpath transmission in optical networks, it is essential to predict the quality of transmission (QoT) of the candidate lightpaths before their deployment. An artificial neural network (ANN) becomes a promising solution for QoT estimation of lightpaths, owing to its powerful data fitting capability and scalability. As a supervised learning model, an ANN requires a large set of training samples from lightpaths’ observations to ensure its accuracy. Unfortunately, the acquisition of the training samples is hindered by practical limitations, such as the shortage of monitorable lightpaths in the early stage of optical network deployment or/and the absence of optical performance monitors at partial network nodes. Therefore, how to obtain a high-precision QoT estimator with small-sized datasets is challenging. In this paper, we propose an evolutionary neuron-level transfer learning (ENTL) scheme for QoT estimation to improve the accuracy of the ANN model with small-sized datasets. In the ENTL-based QoT estimator, the minimal unit of knowledge transfer is the neuron of the ANN model, and the particle swarm optimization (PSO) algorithm is introduced to determine the trainable neurons and the frozen neurons, where the testing dataset generated by data augmentation assists the PSO algorithm to evaluate the feasible solutions. Simulation results show that the ENTL-based QoT estimator achieves higher accuracy than the traditional layer-level transfer learning (LTL)-based QoT estimator. And, when the ENTL-based QoT estimator is applied to optical network planning, it improves the reliability and throughput of optical networks compared with the LTL-based QoT estimator.

Fast QoT estimation method using cascaded artificial neural network for real-time path provisioning in IMDD based all-optical networks.

We propose a fast quality of transmission (QoT) estimation method based on cascaded artificial neural networks (ANNs) for intensity modulation-direct detection (IMDD) systems. The proposed method can calculate the bit error rate of three-span 36 km transmission within 0.7 seconds while taking account of the deterministic waveform distortion caused by the chromatic dispersion and self-phase modulation (SPM).

Improving the accuracy of QoT estimation with insertion loss distribution evaluation for C + L band transmission systems

The accuracy of quality of transmission (QoT) estimation is crucial for software-defined optical networks (SDONs), where network planning and system management of the centralized controller depend on reliable performance evaluation. The insertion loss (IL) commonly exists along the light path between the transmitter side (Tx) and the receiver side (Rx) in fiber optical transmission systems, and the IL values vary for each connector and pigtail. In the case of C+L band transmission systems that are about to be deployed on a large scale, in the presence of severe stimulated Raman scattering (SRS), IL further influences the launch power profile (LPP) as well as the noise spectrum, and the process in the QoT estimation will be disturbed without available IL values. Therefore, there is a great need for an accurate calculation method of IL distribution. In this paper, we propose a method to estimate IL distribution in fiber optical transmission links to improve the accuracy of QoT estimation based on the SRS effect. The results demonstrate that, compared with other common methods, our proposed method greatly improves the accuracy of QoT estimation for C+L band transmission systems with accurate IL distribution evaluation. The high accuracy, robustness, and ability to adapt dynamic conditions make our proposed method a promising QoT estimation scheme in practical C+L band fiber optical communication systems.

Open-source data for QoT estimation in optical networks from Alibaba

There has been a lot of effort in our industry to develop analytical and numerical models for accurate estimation of quality of transmission (QoT), which is essential to efficiently design and operate optical transport networks. However, most researchers do not have access to real optical network data to validate and improve their models. We provide data from our production optical transport network on GitHub, hoping it can be of some help for our community to advance in this area. This letter describes how to use the open-source data.

Experimental Validation of Machine Learning-Based Joint Failure Management and Quality of Transmission Estimation

Experimental Validation of Machine Learning-Based Joint Failure Management and Quality of Transmission Estimation

Comparison of fast quality of transmission estimation methods for C + L + S optical systems

Due to the high potential of multi-band transmission (MBT) systems as a short- to medium-term solution to the ever-increasing demand for fiber capacity, one of the recent areas of research concerns the development of fast but still accurate models to estimate the quality of transmission (QoT) in these systems, since this capability will be paramount for efficient and cost-effective network planning and operation. However, accurately evaluating the QoT of MBT systems with moderate computational complexity is challenging, mainly because stimulated Raman scattering (SRS) becomes a major transmission impairment. Although several models have been proposed in the literature, to the best of the authors’ knowledge, there is a lack of a comprehensive comparison of the advantages and main limitations of the available methodologies. Therefore, this work analyzes the computational time and accuracy of several QoT estimation methods suitable for MBT systems [closed-form interchannel SRS Gaussian noise (ISRS-GN) and four-wave mixing (FWM) models], focusing on transmission systems comprising a total transmission bandwidth of up to 20 THz, which includes not only the C- and L-bands but also a part of the complete S-band. We also propose a modification to the FWM model to take into account the SRS effect in the calculation of the nonlinear interference, referred to as the enhanced FWM (eFWM) model. We show that the closed-form ISRS-GN model has the best trade-off between computational complexity (speed) and accuracy for launch powers per channel up to 4 dBm in a system with 10 spans and with span lengths varying from 50 km to 100 km. However, this method shows generalized signal-to-noise ratio estimation errors as high as 1.5 dB for higher launch powers. Similar conclusions were drawn when using the models to optimize the launch powers in the reference Italian backbone network, i.e., using the closed-form ISRS-GN model leads to more accurate optimized launch powers. However, in this case, the impact on the network-wide average channel capacity of using the launch power optimized using the different approaches is negligible.

Sample-distribution-matching-based transfer learning for QoT estimation in optical networks

Machine learning has been introduced to estimate the quality of transmission (QoT) of lightpaths to meet the reliability of optical network transmission. In the early deployment phase of optical networks, it is difficult to collect enough training samples due to the insufficient lightpaths and monitoring equipment, which makes the estimation models inaccurate. Transfer learning (TL) has been demonstrated as a promising technology for improving the accuracy of estimation models. The main idea of the TL method is pre-training the QoT estimation model with the source domain samples, and then using a few target domain samples to fine-tune the models, which are named fine-tuning samples. However, there are many differences in sample distribution between the source and target domain networks, and more fine-tuning samples to train the TL models are required. Thus, how to improve the accuracy of TL-based QoT estimation models with few samples needs to be examined. This paper proposes a sample-distribution-matching-based transfer learning (SDMbTL) method to perform an accurate QoT estimation with fewer samples. The proposed SDMbTL method designs the sample distribution matching model to filter source domain samples to match the distribution of the target domain samples, which makes the pre-training model more suitable for target networks. We also propose three different matching algorithms to accommodate the discriminative features to be matched. The simulation results demonstrate that the performance of the proposed estimation models outperforms the traditional TL models, saving more than 28.5% of fine-tuning samples.

Improved QoT estimations through refined signal power measurements and data-driven parameter optimizations in a disaggregated and partially loaded live production network

Accurate quality of transmission (QoT) estimations are essential enablers for future low-margin dynamic optical network operations. However, physical parameter measurement uncertainties and other intractable signal propagation effects degrade the accuracy of QoT estimation, especially in live production networks. The recent trend of network disaggregation further exacerbates the issue, and a vendor-agnostic accurate QoT estimator is much needed. In this paper, we study Gaussian-noise-model-based QoT estimation in a large-scale disaggregated and partially loaded live production network with monitored physical layer data spanning across 8 months. We first propose refining the signal power measurements by combining the inline amplifier and optical channel monitor (OCM) power measurements, followed by estimating the gain and noise power profiles of each inline amplifier, which in turn improves QoT estimation accuracy. We further introduce an optical multiplex section and frequency bias to the analytical model to incorporate intractable location-specific and spectral effects in the network and proposed data-driven parameter optimizations to learn the biases as well as erbium-doped fiber amplifier noise figures. The (mean, standard deviation) of the QoT estimation errors were reduced from (−0.1043, 0.6037) dB using average amplifier power and (−0.7875, 0.6337) dB using OCM power to (−0.0964, 0.4649) dB after input parameter refinements and were further reduced to (0.0046, 0.2377) dB with data-driven parameter optimization. The proposed methodologies are simple procedures that network operators can adopt to optimize analytical-model-based QoT estimators and/or serve as feature engineering procedures preceding machine-learning-based QoT in realistic disaggregated live production networks.

Machine learning enhancement of a digital twin for wavelength division multiplexing network performance prediction leveraging quality of transmission parameter refinement

Digital twins capable of quality of transmission (QoT) estimation and prediction are powerful tools for efficient design and operation of optical networks. In this paper, we employ machine learning techniques to enhance both the QoT estimation and prediction capabilities of an optical network digital twin. By leveraging a method to infer or refine the unknown lumped loss distributions and amplifier gain spectra for accurate characterization of the current optical network state, the accuracy of estimation and prediction is substantially improved. For QoT prediction, we further develop a novel neural-network architecture for erbium-doped fiber amplifiers that, after factory training on a single device and with fully loaded configurations only, generalizes to partially loaded configurations seen after deployment and to different gain and tilt settings and other physical devices of the same type. We combine refined parameters and a novel method that predicts the difference in per-channel power when individual services are added or removed or multiple services are lost due to a fiber cut. The impact of error amplification due to cascading of individual components’ models is shown to be reduced, yielding predictions that are substantially more accurate than simply cascaded predictions. As the network ages, the digital twin can be updated by retraining while using only information available in the current network state.

Deep Neural Network-Based QoT Estimation for SMF and FMF Links

Quality of transmission (QoT) estimation tools for fiber links are the enabler for the deployment of reconfigurable optical networks. To dynamically set up lightpaths based on traffic request, a centralized controller must base decisions on reliable performance predictions. QoT estimation methods can be categorised in three classes: exact analytical models which provide accurate results with heavy computations, approximate formulas that require less computations but deliver a reduced accuracy, and machine learning (ML)-based methods which potentially have high accuracy with low complexity. To operate an optical network in real-time, beside accurate QoT estimation, the speed in delivering results is a strict requirement. Based on this, only the last two categories are candidates for this application. In this paper, we present a deep neural network (DNN) structure for QoT estimation considering both regular single-mode fiber (SMF) and future few-mode fiber (FMF) proposed to increase the overall network capacity. We comprehensively explore ML-based regression methods for estimating generalized signal-to-noise ratio (GSNR) in partial-load SMF and FMF links. Synthetic datasets have been generated using the enhanced Gaussian noise (EGN) model. Results indicate that the proposed DNN-based regressor can provide better accuracy along with less computation complexity, compared with other state-of-the-art ML methods as well as closed-form-EGN and closed-form-GN models.

Impact of the Interplay Between Mode Dispersion and Kerr Effect in SDM Optical Networks

Space division multiplexing (SDM) – such as based on multi-mode fibers – is under investigation to support the growing capacity demands. In optical networks, the estimation of quality of transmission (QoT) is a fundamental control task to identify the most suitable transmission parameters (e.g., modulation format) to meet proper spectral efficiency while satisfying the required optical reach. The Gaussian Noise (GN) model is widely adopted for QoT estimation. However, the basic version of the GN model does not account for the interplay between mode dispersion and the Kerr effect. In this paper, we investigate the impact of mode dispersion on SDM networks exploiting strongly coupled modes, through an extended GN model. Network performance analysis is evaluated. First, an analysis of the achievable information rate (AIR) with Gaussian data statistics is presented. Then, a network capacity analysis is shown constrained to a set of supported modulation formats and a fixed symbol rate. Finally, blocking probability is studied with the same set of supported modulation formats with variable symbol rates. The presented network analysis shows that accounting for mode dispersion increases the throughput considering: i) Gaussian data statistics, ii) different modulation formats. Moreover, the blocking probability analysis shows a blocking reduction when mode dispersion is considered in the physical layer modeling.

Iterative supervised learning approach using transceiver bit-error-rate measurements for optical line system optimization

Defining the working points of optical amplifiers is a key factor when managing optical networks, particularly for the quality of transmission (QoT) of deployed connections. However, given the lack of knowledge of physical layer parameters, in many cases operators use these infrastructures suboptimally. In this work, a methodology is presented that optimizes the QoT of an optical line system (OLS) by setting the working points of the erbium-doped fiber amplifiers (EDFAs), by analysis of simulations that use synthetic data derived from experimental characterization of commercial devices. The procedure is divided into three phases: a physical layer characterization, a design process, and an iterative supervised learning approach. Within the first phase, a novel amplifier physical layer characterization is used, exploiting a simple EDFA model that allows an efficient estimation of the OLS behavior, knowing only the setting operative ranges of the devices. The results show that the satisfactory outcome produced during the design phase is further improved by the iterative supervised learning approach. The latter approach is implemented for single OLSs between couples of adjacent reconfigurable optical add and drop multiplexers, each equipped with a certain set of transceivers, enabling the QoT estimation of the specific OLS.

Self-Attention Mechanism-Based Multi-Channel QoT Estimation in Optical Networks

It is essential to estimate the quality of transmission (QoT) of lightpaths before their establishment for efficient planning and operation of optical networks. Due to the nonlinear effect of fibers, the deployed lightpaths influence the QoT of each other; thus, multi-channel QoT estimation is necessary, which provides complete QoT information for network optimization. Moreover, the different interfering channels have different effects on the channel under test. However, the existing artificial-neural-network-based multi-channel QoT estimators (ANN-QoT-E) neglect the different effects of the interfering channels in their input layer, which affects their estimation accuracy severely. In this paper, we propose a self-attention mechanism-based multi-channel QoT estimator (SA-QoT-E) to improve the estimation accuracy of the ANN-QoT-E. In the SA-QoT-E, the input features are designed as a sequence of feature vectors of channels that route the same path, and the self-attention mechanism dynamically assigns weights to the feature vectors of interfering channels according to their effects on the channel under test. Moreover, a hyperparameter search method is used to optimize the SA-QoT-E. The simulation results show that, compared with the ANN-QoT-E, our proposed SA-QoT-E achieves higher estimation accuracy, and can be directly applied to the network wavelength expansion scenarios without retraining.

QoT estimation using EGN-assisted machine learning for multi-period network planning

The rapidly growing traffic demands in fiber-optical networks require flexibility and accuracy in configuring lightpaths, for which fast and accurate quality of transmission (QoT) estimation is of pivotal importance. This paper introduces a machine learning (ML)-based QoT estimation approach that meets these requirements. The proposed gradient-boosting ML model uses precomputed per-channel self-channel-interference values as representative and condensed features to estimate non-linear interference in a flexible-grid network. With an enhanced Gaussian noise (GN) model simulation as the baseline, the ML model achieves a mean absolute signal-to-noise ratio error of approximately 0.1 dB, which is an improvement over the GN model. For three different network topologies and network planning approaches of varying complexities, a multi-period network planning study is performed in which ML and GN are compared as path computation elements (PCEs). The results show that the ML PCE is capable of matching or slightly improving the performance of the GN PCE on all topologies while reducing significantly the computation time of network planning by up to 70%.

Towards explainable artificial intelligence in optical networks: the use case of lightpath QoT estimation

Artificial intelligence (AI) and machine learning (ML) continue to demonstrate substantial capabilities in solving a wide range of optical-network-related tasks such as fault management, resource allocation, and lightpath quality of transmission (QoT) estimation. However, the focus of the research community has been centered on ML models’ predictive capabilities, neglecting aspects related to models’ understanding, i.e., to interpret how the model reasons and makes its predictions. This lack of transparency hinders the understanding of a model’s behavior and prevents operators from judging, and hence trusting, the model’s decisions. To mitigate the lack of transparency and trust in ML, explainable AI (XAI) frameworks can be leveraged to explain how a model correlates input features to its outputs. In this paper, we focus on the application of XAI to lightpath QoT estimation. In particular, we exploit Shapley additive explanations (SHAP) as the XAI framework. Before presenting our analysis, we provide a brief overview of XAI and SHAP, then discuss the benefits of the application of XAI in networking and survey studies that apply XAI to networking tasks. Then, we model the lightpath QoT estimation problem as a supervised binary classification task to predict whether the value of the bit error rate associated with a lightpath is below or above a reference acceptability threshold and train an ML extreme gradient boosting model as the classifier. Finally, we demonstrate how to apply SHAP to extract insights about the model and to inspect misclassifications.

Survey on the Use of Machine Learning for Quality of Transmission Estimation in Optical Transport Networks

Estimating the Quality of Transmission (QoT) of the optical signal from source to destination nodes is the cornerstone of design engineering and service provisioning in optical transport networks. Recent studies have turned to Machine Learning (ML) techniques to improve the accuracy of QoT estimation. In this paper, we survey the literature on this topic and classify the studies into categories based on their scope. Accordingly, we distinguish four categories of ML-based solutions: i) check lightpath feasibility, ii) estimate a lightpath's QoT, iii) enhance existing analytical models and iv) improve model generalization. We describe the proposed solutions in each category in terms of ML algorithms, inputs/outputs of the models, source of data and performance evaluation. Deploying a ML-based solution in the real field is not straightforward and presents several challenges. Therefore, we also discuss from an operator's perspective the potential of these solutions for real-field deployment.

Scalable and Disaggregated GGN Approximation Applied to a C+L+S Optical Network

We investigate quality of transmission (QoT) estimation in a multi-band transmission scenario, including a wideband description of the frequency-dependent physical layer parameters and a disaggregated QoT estimation approach.In particular, we use experimentally-derived erbium- and thulium-doped fiber amplifier operational data, and simulate the generation of the nonlinear interference (NLI) noise by comparing implementations of the split-step Fourier method, the generalized Gaussian noise (GGN) model implemented in the open-source GNPy library, and an efficient approximation that provides accurate results in a limited computational time. The latter semi-analytical solution is obtained by approaching the Manakov equation from a disaggregated standpoint, including the power transfer induced by stimulated Raman scattering, along with the frequency-dependent evolution of the fiber parameters of loss, dispersion and the nonlinearity coefficient. We validate the proposed approximation within a C+L+S 400 G transmission scenario and investigate 21 channels under test that are equally spaced along the three bands.

A Meta-Learning-Assisted Training Framework for Physical Layer Modeling in Optical Networks

An accurate quality of transmission (QoT) estimation can help to reduce the <i>design margin</i> of optical network planning. The physical layer impairments (PLI) modeling can be the basis of QoT estimation and support upper layer applications. Due to the complex structure of the physical layer and imperfect knowledge of network parameters, improving PLI modeling performance is hindered by the parameter uncertainty. The parameter uncertainty can be alleviated by online adaptation using data from the real system. However, obtaining the accurate value of each PLI is difficult since multiple impairments co-exist in the optical system. Therefore, updating PLI models with online adaptation is difficult, which requires PLI models with higher robustness to parameter uncertainty and more efficient adaptation ability with limited data from real systems. In this paper, we propose a meta-learning-assisted training framework for machine-learning-based physical layer models. This framework can improve the model robustness to agnostic uncertain parameters during offline training and enables the model to efficiently adapt to the real system with fewer data. Use cases of dealing with parameter uncertainty in fiber nonlinearity modeling and QoT estimation are performed to prove the concept through simulations, which demonstrates the robustness improvement and <i>few-shot learning</i> ability.