Ground Truth Test Research Articles

Repurposing non-pharmacological interventions for Alzheimer's disease through link prediction on biomedical literature

Non-pharmaceutical interventions (NPI) have great potential to improve cognitive function but limited investigation to discover NPI repurposing for Alzheimer's Disease (AD). This is the first study to develop an innovative framework to extract and represent NPI information from biomedical literature in a knowledge graph (KG), and train link prediction models to repurpose novel NPIs for AD prevention. We constructed a comprehensive KG, called ADInt, by extracting NPI information from biomedical literature. We used the previously-created SuppKG and NPI lexicon to identify NPI entities. Four KG embedding models (i.e., TransE, RotatE, DistMult and ComplEX) and two novel graph convolutional network models (i.e., R-GCN and CompGCN) were trained and compared to learn the representation of ADInt. Models were evaluated and compared on two test sets (time slice and clinical trial ground truth) and the best performing model was used to predict novel NPIs for AD. Discovery patterns were applied to generate mechanistic pathways for high scoring candidates. The ADInt has 162,212 nodes and 1,017,284 edges. R-GCN performed best in time slice (MR = 5.2054, Hits@10 = 0.8496) and clinical trial ground truth (MR = 3.4996, Hits@10 = 0.9192) test sets. After evaluation by domain experts, 10 novel dietary supplements and 10 complementary and integrative health were proposed from the score table calculated by R-GCN. Among proposed novel NPIs, we found plausible mechanistic pathways for photodynamic therapy and Choerospondias axillaris to prevent AD, and validated psychotherapy and manual therapy techniques using real-world data analysis. The proposed framework shows potential for discovering new NPIs for AD prevention and understanding their mechanistic pathways.

In the Quest for Scale-optimal Mappings

Optimal mapping is one of the longest-standing problems in computational mathematics. It is natural to measure the relative curve length error under map to assess its quality. The maximum of such error is called the quasi-isometry constant, and its minimization is a nontrivial max-norm optimization problem. We present a physics-based quasi-isometric stiffening (QIS) algorithm for the max-norm minimization of hyperelastic distortion. QIS perfectly equidistributes distortion over the entire domain for the ground-truth test (unit hemisphere flattening) and, when it is not possible, tends to create zones where all cells have the same distortion. Such zones correspond to fragments of elastic material that became rigid under stiffening, reaching the deformation limit. As such, maps built by QIS are related to the de Boor equidistribution principle, which asks for an integral of a certain error indicator function to be the same over each mesh cell. Under certain assumptions on the minimization toolbox, we prove that our method can build, in a finite number of steps, a deformation whose maximum distortion is arbitrarily close to the (unknown) minimum. We performed extensive testing: on more than 10,000 domains QIS was reliably better than the competing methods. In summary, we reliably build 2D and 3D mesh deformations with the smallest known distortion estimates for very stiff problems.

Imaging Lunar Craters with the Lucy Long Range Reconnaissance Imager (L’LORRI): A Resolution Test for NASA's Lucy Mission

NASA's Lucy mission is designed to better understand the unique population of Trojan asteroids. Trojans were probably captured in Jupiter's L4 and L5 points early in the solar system's evolution and little altered since then. A critical investigation of Lucy is to use its highest-resolution camera, the Lucy Long Range Reconnaissance Imager (L’LORRI), to image Trojans’ surfaces to understand their geology and impact crater populations. Through crater statistics, the population of smaller bodies that produced those impacts, relative age differences across the bodies, and other comparative investigations between bodies can be studied. Mapping the crater population to the minimum diameters needed to achieve Lucy's objectives might require image subsampling and deconvolution (“processing”) to improve the spatial resolution, a process whereby multiple, slightly offset images are merged to create a single, better-sampled image and deconvolved with L’LORRI's point-spread function. Lucy's first Earth Gravity Assist (EGA1) provided an opportunity to test this process's accuracy using L’LORRI images of the Moon, whose crater population is well characterized and therefore provides ground-truth testing. Specifically, the lunar crater imaging by L’LORRI during EGA1 allowed us to compare crater statistics derived from raw and processed L’LORRI images with ground-truth statistics derived from higher-resolution lunar imaging from other missions. The results indicate the processing can improve impact crater statistics such that features can be identified and measured to ≈70% the diameter that they can otherwise be reliably mapped on native L’LORRI images. This test's results will be used in the observation designs for the Lucy flyby targets.

“Extreme Highest” and “Extreme Anomalous”: Proposed indices for chlorophyll-a extreme events in European seas between 2003 and 2021

This study addresses the occurrence of extreme values of chlorophyll-a concentrations in European seas, using the satellite's OBGP MODIS AQUA v2018 dataset, for the 2003–2021 period. Two novel and complementary statistical indices based on the combination of the overall period's and the monthly 90th percentiles (P90 and mP90, respectively) are proposed: (i) the Extreme Highest exceedances (EH) include the observations with highest chlorophyll-a magnitudes in each location (larger than P90 and mP90); (ii) the Extreme Anomalous exceedances (EA) address the observations larger than mP90 but lower than P90, thus representing relevant anomalies during low phytoplankton growing seasons. Given the important differences of available observations per pixel, the EH and EA exceedances are normalized and also presented in percentage units.Although the occurrence of these indices greatly varies in time and space, the aggregated statistics and maps reveal some clear patterns: EH and EA have very distinct (almost complementary) seasonal and spatial distribution, EH prevail in mesotrophic and euphotic waters during the main interannual bloom season whilst EA are more abundant in oligotrophic waters out of the main seasonal bloom in each area. Both significant (p < 0.05) increasing and decreasing annual trends have been found in different European Seas. Overall, these EH and EA trends are reflecting the climate-driven physical and ecological changes in European Seas. Although these results and the conceptual and computational simplicity of these indices are encouraging, further ground truth testing is required to account for their uncertainties, mostly related to data representativeness and the performance of the chlorophyll-a estimation algorithms.

Field-scale dynamics of planting dates in the US Corn Belt from 2000 to 2020

Crop planting dates are a dynamic feature of agricultural systems that respond to short- and long-term climate signals, crop and cultivar selection, and technology changes. Planting date records are essential for yield gap analyses, accurate crop modeling, and tracking farmer adaptations to weather and climate change. Although planting dates have high variation at local scales due to heterogeneity in farm resources and decision-making, available long-term data on planting dates are largely restricted to aggregated regional statistics or, at best, satellite-derived datasets with limited spatiotemporal extent and at resolutions unable to distinguish individual fields. Here, we generated retrospective annual field-scale (30 m) planting date maps for both maize and soybeans spanning 2000–2020 across a 12-state region in the United States Corn Belt based on Landsat satellite data and a large ground sample of over 28,000 maize and soybean fields. Using training data from 2015 to 2020 for model selection, we found that planting date predictions improved with harmonic regression of Landsat data and additional annual weather covariates. The preferred random forest model approximately doubled performance compared to a null model based on state median planting dates, capturing 47% of field-level variation for maize (mean absolute error, MAE = 7.4 days) and 44% for soybeans (MAE = 7.5 days) against held-out ground truth test data for 2008–2014. We also evaluated the full 2000–2020 dataset with state agricultural statistics, finding strong agreement with median planting dates for maize (R2 = 0.76, MAE = 4.4 days) and slightly lower agreement for soybeans (R2 = 0.65, MAE = 5.4 days) when aggregated to the state level. We then used this new dataset to analyze environmental determinants of planting dates at a finer-scale than previously possible, controlling for unobserved variation at the sub-state district level. We found that during 2000–2020, each standard deviation increase in early-season rainfall delayed planting by ∼2.5 days, and fields with higher soil productivity ratings tended to be planted earlier. We did not find meaningful trends over the last two decades in planting dates for maize or soybeans, in contrast to trends towards earlier planting dates late last century and predicted for this period in climate adaptation studies. We hypothesize increases in early season rainfall or persisting early-season frost risks may have inhibited these shifts towards earlier planting. Remotely sensed planting dates will be a useful tool for yield gap analyses, crop simulation modeling, and ongoing assessment of climate adaptation.

Finite element model updating with quantified uncertainties using point cloud data

AbstractWhile finite element (FE) modeling is widely used for ultimate strength assessments of structural systems, incorporating complex distortions and imperfections into FE models remains a challenge. Conventional methods typically rely on assumptions about the periodicity of distortions through spectral or modal methods. However, these approaches are not viable under the many realistic scenarios where these assumptions are invalid. Research efforts have consistently demonstrated the ability of point cloud data, generated through laser scanning or photogrammetry-based methods, to accurately capture structural deformations at the millimeter scale. This enables the updating of numerical models to capture the exact structural configuration and initial imperfections without the need for unrealistic assumptions. This research article investigates the use of point cloud data for updating the initial distortions in a FE model of a stiffened ship deck panel, for the purposes of ultimate strength estimation. The presented approach has the additional benefit of being able to explicitly account for measurement uncertainty in the analysis. Calculations using the updated FE models are compared against ground truth test data as well as FE models updated using standard spectral methods. The results demonstrate strength estimation that is comparable to existing approaches, with the additional advantages of uncertainty quantification and applicability to a wider range of application scenarios.

Convolutional Neural Networks with Mask Shift Loss for Improving Building Outlines Detection

In the development of supervised semantic segmentation models for building outline detection, a significant challenge is obtaining sufficient accurately labelled training data. Old building outline datasets captured against historic imagery often exist covering the same locations as new imagery. By training Deep Convolutional Neural Networks to recognise buildings in these areas we can detect new and changed buildings. Ideally, historic building outlines as training labels paired with new orthophotography could be used to train models to detect buildings in new imagery. However, historic imagery may be orthorectified to a different standard, or have different off-nadir angles resulting in an apparent displacement of buildings between historic and current imagery sets and in labelled buildings that do not align accurately with the building image. This displacement problem presents difficulties in training accurate models. This paper proposes a novel loss function that searches the prediction segmentation masks for the best matching crop to a central crop of the training segmentation mask, thus reducing error due to linear displacements between an image and mask. According to the Intersection-over-Union metric, a ground truth test set for the building outlines detection problem improved from 0.751 to 0.802 on the Christchurch Buildings dataset when using the mask shift loss versus binary cross-entropy.

Enhanced Automatic Morphometry of Nerve Histological Sections Using Ensemble Learning

There is a need for an automated morphometry algorithm to facilitate the otherwise labor-intensive task of the quantitative histological analysis of neural microscopic images. A benchmark morphometry algorithm is the convolutional neural network Axondeepseg (ADS), which yields a high segmentation accuracy for scanning and transmission electron microscopy images. Nevertheless, it shows decreased accuracy when applied to optical microscopy images, and it has been observed to yield sizable false positives when identifying small-sized neurons within the slides. In this study, ensemble learning is used to enhance the performance of ADS by combining it with the paired image-to-image translation algorithm PairedImageTranslation (PIT). Here, 120 optical microscopy images of peripheral nerves were used to train and test the ensemble learning model and the two base models individually for comparison. The results showed weighted pixel-wise accuracy for the ensemble model of 95.5%, whereas the ADS and PIT yielded accuracies of 93.4% and 90%, respectively. The automated measurements of the axon diameters and myelin thicknesses from the manually marked ground truth images were not statistically different (p = 0.05) from the measurements taken from the same images when segmented using the developed ensemble model, while they were different when they were measured from the segmented images by the two base models individually. The automated measurement of the G ratios indicated a higher similarity to the ground truth testing images for the ensemble model in comparison with the individual base models. The proposed model yielded automated segmentation of the nerve slides, which were sufficiently equivalent to the manual annotations and could be employed for axon diameters and myelin thickness measurements for fully automated histological analysis of the neural images.

Detection of Sugarcane Mosaic Diseases Using Deep Learning Architecture to Avoid Annealing Temperature of PCR Primer in Laboratory Testing

The sugarcane leaf diseases such as mosaic and streak mosaic are difficult to differentiate using Image processing techniques because both diseases show similar visual attributes such as pattern and color. To identify the type of diseases, we need to perform Polymerase Chain Reaction (PCR) testing which is used for the classification of diseases in laboratories. The accuracy of the PCR test depends on reaction mix preparation, reaction time, and DNA/RNA extraction. The major problem influencing the PCR test accuracy is the Annealing temperature of the primers and needs a standardized set of samples. In addition, it is a time-consuming process. In this paper, we proposed a Diversified Deep Learning Architecture (DDLA) which is developed with the input images after various pre-processing steps such as denoising using Discrete Wavelet Transform (DWT) and enhancing using histogram equalization in HSI color space to improve the similar pattern disease prediction accuracy. The performance of the proposed model is analyzed for a set of diseased leaves and the results are compared with the output of the popular pertained models such asVGG16, InceptionV3, ResNET50, Inception ResNET, and DenseNET201 with and without pre-processing. The training accuracy of the proposed model is 97% and the testing accuracy is 87%. The DDLA model produces ground truth test results with an accuracy of 88.7% for mosaic and 85.7% for streak mosaic with a less computational time of 152sec compared to the lab test duration of 6 hrs. The performance of the model is also measured in terms of Precision, F1 Score, Specificity, and Sensitivity. The Proposed DDLA model’s F1 score is higher than the pre-trained models with a minimum test loss of 1.167. Moreover, the DDLA structure occupies less memory space when compared to the pertained models.

CIRO: The Effects of Visually Diminished Real Objects on Human Perception in Handheld Augmented Reality

Augmented reality (AR) scenes often inadvertently contain real world objects that are not relevant to the main AR content, such as arbitrary passersby on the street. We refer to these real-world objects as content-irrelevant real objects (CIROs). CIROs may distract users from focusing on the AR content and bring about perceptual issues (e.g., depth distortion or physicality conflict). In a prior work, we carried out a comparative experiment investigating the effects on user perception of the AR content by the degree of the visual diminishment of such a CIRO. Our findings revealed that the diminished representation had positive impacts on human perception, such as reducing the distraction and increasing the presence of the AR objects in the real environment. However, in that work, the ground truth test was staged with perfect and artifact-free diminishment. In this work, we applied an actual real-time object diminishment algorithm on the handheld AR platform, which cannot be completely artifact-free in practice, and evaluated its performance both objectively and subjectively. We found that the imperfect diminishment and visual artifacts can negatively affect the subjective user experience.

A Multi-Layer Perceptron Network for Perfusion Parameter Estimation in DCE-MRI Studies of the Healthy Kidney

Background: Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is an imaging technique which helps in visualizing and quantifying perfusion—one of the most important indicators of an organ’s state. This paper focuses on perfusion and filtration in the kidney, whose performance directly influences versatile functions of the body. In clinical practice, kidney function is assessed by measuring glomerular filtration rate (GFR). Estimating GFR based on DCE-MRI data requires the application of an organ-specific pharmacokinetic (PK) model. However, determination of the model parameters, and thus the characterization of GFR, is sensitive to determination of the arterial input function (AIF) and the initial choice of parameter values. Methods: This paper proposes a multi-layer perceptron network for PK model parameter determination, in order to overcome the limitations of the traditional model’s optimization techniques based on non-linear least-squares curve-fitting. As a reference method, we applied the trust-region reflective algorithm to numerically optimize the model. The effectiveness of the proposed approach was tested for 20 data sets, collected for 10 healthy volunteers whose image-derived GFR scores were compared with ground-truth blood test values. Results: The achieved mean difference between the image-derived and ground-truth GFR values was 2.35 mL/min/1.73 m2, which is comparable to the result obtained for the reference estimation method (−5.80 mL/min/1.73 m2). Conclusions: Neural networks are a feasible alternative to the least-squares curve-fitting algorithm, ensuring agreement with ground-truth measurements at a comparable level. The advantages of using a neural network are twofold. Firstly, it can estimate a GFR value without the need to determine the AIF for each individual patient. Secondly, a reliable estimate can be obtained, without the need to manually set up either the initial parameter values or the constraints thereof.

Automated Curation of CNMF-E-Extracted ROI Spatial Footprints and Calcium Traces Using Open-Source AutoML Tools.

In vivo 1-photon (1p) calcium imaging is an increasingly prevalent method in behavioral neuroscience. Numerous analysis pipelines have been developed to improve the reliability and scalability of pre-processing and ROI extraction for these large calcium imaging datasets. Despite these advancements in pre-processing methods, manual curation of the extracted spatial footprints and calcium traces of neurons remains important for quality control. Here, we propose an additional semi-automated curation step for sorting spatial footprints and calcium traces from putative neurons extracted using the popular constrained non-negative matrixfactorization for microendoscopic data (CNMF-E) algorithm. We used the automated machine learning (AutoML) tools TPOT and AutoSklearn to generate classifiers to curate the extracted ROIs trained on a subset of human-labeled data. AutoSklearn produced the best performing classifier, achieving an F1 score >92% on the ground truth test dataset. This automated approach is a useful strategy for filtering ROIs with relatively few labeled data points and can be easily added to pre-existing pipelines currently using CNMF-E for ROI extraction.

A validation framework for neuroimaging software: The case of populationreceptive fields.

Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validity of the results. It is difficult, nay impossible, for researchers to check the accuracy of software by reading the source code; ground truth test datasets are needed. Computational reproducibility means providing software so that for the same input anyone obtains the same result, right or wrong. Computational validity means obtaining the right result for the ground-truth test data. We describe a framework for validating and sharing software implementations, and we illustrate its usage with an example application: population receptive field (pRF) methods for functional MRI data. The framework is composed of three main components implemented with containerization methods to guarantee computational reproducibility. In our example pRF application, those components are: (1) synthesis of fMRI time series from ground-truth pRF parameters, (2) implementation of four public pRF analysis tools and standardization of inputs and outputs, and (3) report creation to compare the results with the ground truth parameters. The framework was useful in identifying realistic conditions that lead to imperfect parameter recovery in all four pRF implementations, that would remain undetected using classic validation methods. We provide means to mitigate these problems in future experiments. A computational validation framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely upon a few agreed upon packages. We hope that the framework will be helpful to validate other critical neuroimaging algorithms, as having a validation framework helps (1) developers to build new software, (2) research scientists to verify the software's accuracy, and (3) reviewers to evaluate the methods used in publications and grants.

Domain‐specific visualization system based on automatic multiseed recommendations: Extracting stratigraphic structures

SummaryUnderground flow path (UFP) is one of the most significant stratigraphic structures in revealing the distribution of oil or gas from seismic data. We design a domain‐specific visualization system to extract the stratigraphic structures by seed point tracing and explore the seismic data by graph interactions. The seeds are automatically generated by kernel function–based density gradients computation. Users are allowed to adjust the recommended seeds by fine‐tuning them with visual interactions. The seeds are further merged by a weighted quick‐union algorithm to get the link information to construct a graph. Different types of nodes in the graph are designed to enable users to explore the extracted UFP structures intuitively. Finally, we evaluated the proposed approach by performance tests, sensitivity tests, and ground truth tests. The feedback from the domain experts demonstrates that the proposed visualization tool improved the capability of revealing the distribution and geostructures of UFPs compared with the existing methods.

How Does Knowledge of the AUC Constrain the Set of Possible Ground-Truth Labelings?

Recent work on privacy-preserving machine learning has considered how datamining competitions such as Kaggle could potentially be “hacked”, either intentionally or inadvertently, by using information from an oracle that reports a classifier’s accuracy on the test set (Blum and Hardt 2015; Hardt and Ullman 2014; Zheng 2015; Whitehill 2016). For binary classification tasks in particular, one of the most common accuracy metrics is the Area Under the ROC Curve (AUC), and in this paper we explore the mathematical structure of how the AUC is computed from an n-vector of real-valued “guesses” with respect to the ground-truth labels. Under the assumption of perfect knowledge of the test set AUC c=p/q, we show how knowing c constrains the set W of possible ground-truth labelings, and we derive an algorithm both to compute the exact number of such labelings and to enumerate efficiently over them. We also provide empirical evidence that, surprisingly, the number of compatible labelings can actually decrease as n grows, until a test set-dependent threshold is reached. Finally, we show how W can be efficiently whittled down, through pairs of oracle queries, to infer all the groundtruth test labels with complete certainty.

A novel relative homogeneity thresholding method with optimization strategy

Determining thresholds by measuring class variance is highly effective for image segmentation. Otsu’s method and its derivatives are common approaches that are both simple and adaptable. In spite of these methods’ excellent segmentation performance, images with particular gray distributions cause a thresholding bias that limits their usefulness. We explore the limitations of Otsu’s method and apply other evaluation criteria. In particular, we determine the relative homogeneity between the object and the background and then use it as a classification criterion along with a new binary thresholding method. Our method employs a histogram-smoothing method to improve valley-point selection, establishes a uniformity measure to identify the region with the best homogeneity, and identifies an optimization function for obtaining the best values for the adjustable parameters and threshold value. We also introduce a multilevel thresholding criterion based on a binary thresholding approach. Experiments using real and ground truth test images confirm the validity of our proposed method. Our method also offers a denoising ability when configured to use neighborhood information.

Text as Policy: Measuring Policy Similarity through Bill Text Reuse

AbstractThe identification of substantively similar policy proposals in legislation is important to scholars of public policy and legislative politics. Manual approaches are prohibitively costly in constructing datasets that accurately represent policymaking across policy domains, jurisdictions, or time. We propose the use of an algorithm that identifies similar sequences of text (i.e., text reuse), applied to legislative text, to measure the similarity of the policy proposals advanced by two bills. We study bills from U.S. state legislatures. We present three ground truth tests, applied to a corpus of 500,000 bills. First, we show that bills introduced by ideologically similar sponsors exhibit a high degree of text reuse, that bills classified by the National Conference of State Legislatures as covering the same policies exhibit a high degree of text reuse, and that rates of text reuse between states correlate with policy diffusion network ties between states. In an empirical application of our similarity measure, we find that Republican state legislators introduce legislation that is more similar to legislation introduced by Republicans in other states, than is legislation introduced by Democratic state legislators to legislation introduced by Democrats in other states.

Text as Policy: Measuring Policy Similarity Through Bill Text Reuse

The identification of substantively similar policy proposals in both proposed and adopted legislation is important to scholars of public policy diffusion and legislative politics. Conventional, manual, approaches are prohibitively costly in constructing datasets that accurately represent policymaking across policy domains, geographic units, and/or time. We propose the use of text-sequencing algorithms, applied to legislative text, to identify bills that introduce similar policy proposals. We present three ground truth tests, applied to a corpus of 500,000 bills from US-state legislatures. First, we show that bills introduced by ideologically similar sponsors are more likely to exhibit a high degree of text reuse. Second, we show that bills classified by the National Council of State Legislatures as covering the same policies exhibit a high degree of text reuse. Third, we show that rates of text reuse across state borders correlate strongly with the diffusion networks recently introduced by Desmarais, Harden and Boehmke (2015).

Vegetation classification of Coffea on Hawaii Island using WorldView-2 satellite imagery

Coffee is an important crop in tropical regions of the world; about 125 million people depend on coffee agriculture for their livelihoods. Understanding the spatial extent of coffee fields is useful for management and control of coffee pests such as Hypothenemus hampei and other pests that use coffee fruit as a host for immature stages such as the Mediterranean fruit fly, for economic planning, and for following changes in coffee agroecosystems over time. We present two methods for detecting Coffea arabica fields using remote sensing and geospatial technologies on WorldView-2 high-resolution spectral data of the Kona region of Hawaii Island. The first method, a pixel-based method using a maximum likelihood algorithm, attained 72% producer accuracy and 69% user accuracy (68% overall accuracy) based on analysis of 104 ground truth testing polygons. The second method, an object-based image analysis (OBIA) method, considered both spectral and textural information and improved accuracy, resulting in 76% producer accuracy and 94% user accuracy (81% overall accuracy) for the same testing areas. We conclude that the OBIA method is useful for detecting coffee fields grown in the open and use it to estimate the distribution of about 1050 hectares under coffee agriculture in the Kona region in 2012.

Estimation of edge displacement against brightness and camera‐to‐object distance

This study proposes a systematic method to estimate edge displacement (ED) from ground truth edges. To obtain ground truth edges and test edges, a series of target sheets containing the reference regions and brightness contrast regions were designed. Further, the brightness contrast regions were designed to contain foreground and background brightness between which the image edges occur. By varying the foreground brightness (FB), the influence of the FB on the edge locations was tested. Moreover, the influence of camera-to-object distance on the edge locations was tested. A simple least-squares method using slope profiles and a slope-based weighting scheme is proposed for calculating the edges occurring in contrast regions with subpixel accuracy. The experiment results revealed that a bright foreground moves the edge location towards the dark side of the edges in the range of 0-0.6 pixels with strong relationships between the FB and magnitude of ED.