Hierarchical Clustering Procedure Research Articles

Climate model selection via conformal clustering of spatial functional data

AbstractClimate model selection stands as a critical process in climate science and research. It involves choosing the most appropriate climate models to address specific research questions, simulating climate behaviour, or making projections about future climate conditions. This paper proposes a new approach, using spatial functional data analysis, to asses which of the 18 EURO CORDEX simulation models work better for predicting average temperatures in the Campania region (Italy). The method involves two key steps: first, using functional data analysis to process climate variables and select optimal models by a hierarchical clustering procedure; second, validating the chosen models by proposing a new conformal prediction approach to the anomalies associated to each cluster.

Look-Ahead Unit Commitment With Adaptive Horizon Based on Deep Reinforcement Learning

The highly variable nature of renewable energy has led to the concept of intra-day look-ahead unit commitment (LAUC), which aims to determine the on/off status and power outputs of generating units in a rolling-horizon fashion. LAUC is traditionally performed based on look-ahead horizon (LAH) with fixed length and resolution. Such practice can neither capture the high-risk time periods, nor achieve maximum computational efficiency. To address these issues, this paper proposes a LAUC method with adaptive horizon (LAUC-AH). The method uses three parameters to describe the shape of LAH, namely length, resolution, and myopia. Taking these parameters, the forecast profile of renewable energy and load demand is aggregated using a hierarchical clustering procedure to form the LAH, which is then used to construct the LAUC optimization model. Furthermore, a deep reinforcement learning-based agent is used to dynamically adjust the parameters of LAH, such that the LAUC model can adapt to different operation statuses of the power grid. The case studies are carried out on a modified IEEE-118 test case to validate the feasibility and effectiveness of the proposed method.

Monthly Global Solar Radiation Model Based on Artificial Neural Network, Temperature Data and Geographical and Topographical Parameters: A Case Study in Spain

Solar energy plays an essential role in the current energy context to achieve sustainable development while supplying energy needs, creating jobs, and protecting the environment. Many solar radiation models have provided valid estimates at many different locations, using appropriate input variables for specific climatic conditions, but predictions are less accurate on a regional scale. Since radiometric weather stations are relatively dispersed, even in the most developed countries, it is interesting to develop indirect models based on measurements that are common in secondary network stations. This paper develops a monthly global solar radiation model based on a simple neural network structure, using temperature, geographical, and topographical data from 105 meteorological stations, representative of the whole of peninsular Spain. A hierarchical clustering procedure was employed to select the data used to train and validate the model. To avoid functional dependencies between parameters and variables, which hinder the generality of the model, all input and output variables are dimensionless. The estimates fit the 1260 monthly data with RRMSE values of about 6%, which improves results obtained previously, using regression models, and proves that simplicity is compatible with the generality and accuracy of a model, even in large regions with very varied characteristics.

An Examination of Aggregate Side-Effects Mapped to Therapies Experienced By Patients with Relapsed Large B-Cell Lymphomas - an Analysis from the Lymphoma Coalition's 2022 Global Patient Survey

Introduction The identification of side-effects associated with a particular therapy relies on univariable assessments of incidence across treatment arms. While this approach serves to set the groundwork for the establishment of causation, it is suboptimal when the incidence is small or when novel therapeutics with limited deployment are being examined. In this study, we used a multivariate approach to aggregate multiple symptoms in order to segment our sample into those who had differing degrees of side-effect incidence. We used the resultant strata to examine the use of various therapies. Methods Patients with relapsed large B-cell lymphomas who responded to the Lymphoma Coalition's 2022 Global Patient Survey (N=215) were asked if they had ever been treated with a particular treatment (N = 20) and what side effects (N=32) they have ever experienced. Unsupervised hierarchical clustering utilizing Ward's algorithm was used to aggregate the incidence of side-effects (N = 32). Resultant strata from the hierarchical clustering procedure were subjected to contingency analysis with the use of 20 treatments. Differences were assessed with odds ratio's, 95% confidence intervals and p-values from the likelihood chi-square statistic as appropriate. Results The total number of respondents meeting the criteria for inclusion in this study was 215. The median age of respondents was 58 and ranged from 23-85 years. Females comprised 55% of the study sample. Figure 1 Illustrates the two-way clustering heat map. A subjective decision was made to segment the study sample at the first branch yielding two clusters: 1 (N = 76) and 2 (N = 139). The rationale for selecting two clusters was based upon an obvious difference in side-effect incidence. The cumulative incidence of side effects in Cluster 1 = 1570 yielding an incidence rate per patient of 11.3 which is contrasted with Cluster 2 having a cumulative side-effect incidence of 272 yielding a rate per patient of 3.6. Table 1 illustrates the odds ratios and 95% confidence intervals derived from contingency analysis utilizing Cluster as the explanatory variable. Significantly increased odds of CAR-T (p<0.0001), steroids (0.006) and radiation therapy (p = 0.01) use were found in Cluster 1 relative to Cluster 2. We were unable to detect a significant change in the odds ratio for the other 17 treatments. Conclusions These results suggest that there is an association of increased side effects for those receiving CAR-T. Additionally, radiotherapy was also found to be marginally more used in those with more side-effects. The fact that steroid use was also increased in this cluster of patients makes sense since that is a routine prophylaxis strategy for patients experiencing substantial side effects. From the heat map, we can infer two things. First, there are groups of side effects that tend to cluster together which may reflect the treatment heterogeneity provided to this population. Second, it appears that about one-third of these patients experience very few side effects which supports the idea of patient heterogeneity. There are some limitations to this work which includes the use of patient reported outcomes and the fact that we were unable to explicitly map the treatment course of every individual patient. However, this study does support the idea that the diversity of the side-effect profile for CAR-T may extend well beyond the traditional phenomena of cytokine release syndrome, blood, and neurological effects. We suspect that as the use of CAR-T expands, additional side-effects will become apparent. Future studies should focus on identifying individuals who are likely to experience significant side effects and map treatment strategies to the appropriate patients.

Re-defining the Climatic Zones over Nigeria

Aims: This study aimed at identifying climatic zones over Nigeria.
 Duration of Study: Daily air temperature data for the 36 states and FCT in Nigeria were obtained from National Aeronautic and Space Administration (NASA) for the period of thirty-seven (37) years (1984-2020).
 Method: Several hierarchical clustering procedures–single linkages, complete linkage, average distance within clusters, average distance between clusters, centroid clustering, median linkage and Ward’s method were used in this study.
 Results: Based on the findings, median linkage, complete linkage and Ward’s clustering method solutions seems more realistic. The application of cluster analysis, revealed five climate zones (cluster) over Nigeria. Cluster 1 covers Plateau state only while cluster 2 covers three south-west states and one north-west state (Kaduna). Cluster 3 covers all the states in south-east, south-south (except Cross River), North-central (except Kogi and Kwara), Bauchi in North-west and ogun and Oyo in South-west. Cluster 4 covers one state in south-south (Cross River), South-south (Lagos), two states in North-central (Kogi and Kwara), three states in North-east (Adamawa, Gombe and Taraba) and four states in North-west (Jigawa, Kano, Kastina and Zamfara). Lastly cluster 5 covers two states in North-east (Borno and Yobe) and North-west (Kebbi and Sokoto).
 Conclusion: Our findings clearly show that cluster analysis can be applied to identify similar weather/climate state from air temperature data.

Expectations for Children with Autism Spectrum Disorders or Intellectual Disabilities in Ghana: A Comparison Between Service Providers and Parents.

Little is known about care providers' expectations for children with autism spectrum disorders (ASD) and intellectual disabilities (ID) in Ghana. This study used group concept mapping (n = 9) and a quantitative survey (n = 128) to explore and compare service providers' and parents' expectations for children with ASD or ID. Data were analyzed using hierarchical clustering procedures and Multivariate Analysis of Variance (MANOVA). Concept mapping results revealed several expectation clusters, including independence, love and acceptance, equal social rights and opportunities, and professional and caregiver training. MANOVA results revealed significant differences between parents, teachers, and healthcare providers in their perceptions of the importance and likelihood of a child achieving these expectations. Results are discussed in reference to the cultural context, and implications are outlined.

Exploiting Surrogate Safety Measures and Road Design Characteristics towards Crash Investigations in Motorway Segments

High quality data on road crashes, road design characteristics, and traffic are typically required to predict crash frequency. Surrogate Safety Measures (SSMs) are an alternative category of indicators that can be used in road safety analyses in order to quantify various unsafe traffic events. The objective of this research is to exploit road geometry data and SSMs toward various road crash investigations in motorway segments. To that end, for this analysis, a database containing data on injury and property-damage-only crashes, road design characteristics, and SSMs of 668 segments was compiled and utilized. The results of the developed negative binomial regression model revealed that crash frequency is positively correlated with the average annual daily traffic volume, the length of the segment, harsh accelerations, and harsh braking. Moreover, four distinct clusters representing crash risk levels of the examined segments emerged from the hierarchical clustering procedure, ranging from more risk-prone, potentially unsafe locations to more safe locations. These four clusters also formed the response variable classes of a random forest model. This classification model used various road geometry data and SSMs as predictors and achieved high classification performance for all classes, averaging more than 88% correct classification rates.

Detecting Regimes of Economic Growth With Fuzzy Concept-Based Models

The paper presents a new method to detect economic growth regimes based on time-series similarity analysis. The procedure builds upon a fuzzy concept-based model, the Jaccard Index, and a hierarchical clustering procedure. Having introduced the proposed method, we apply it to the time series of economic growth and its proximate factors. Further, we study the stability of the obtained results under different parameters. The proposed procedure allows us to detect groups of countries that share a common growth performance and akin growth mechanisms. Simultaneously, the method let us identify countries that escape any fixed classifications.

Geographic and climatic constraints on bioregionalization of European ants

AbstractAimBiogeographical regionalization is scant for most insect groups due to shortfalls in distribution and phylogenetic information, namely the Wallacean and Darwinian shortfalls respectively. Here, we focused on the European ants and compared new techniques to classical analyses based on regional lists and taxonomic methods. We asked the following: (1) Can grid‐based regionalizations using novel distribution data improve biogeographical transitions? and (2) Can phylogenetic approaches reveal new insights regarding ant evolutionary history?LocationEurope and Anatolia.TaxonAnts (Formicidae).MethodsFirst, we developed a refined database integrating the occurrences of 747 ant species across 207 regions of Europe and Anatolia, based on newly expert‐validated records derived from the existing Global Ant Biodiversity Informatics database. Using range estimates for these species derived from polygons and species distribution modelling, we produced species assemblages in 50 × 50 km grid cells. We calculated taxonomic and phylogenetic turnover of ant assemblages, then performed a hierarchical clustering procedure to delineate biogeographical structure.ResultsAt both the regional list and grid assemblage levels, the Mediterranean has higher turnover and more biogeographical regions than northern Europe, both taxonomically and phylogenetically. Delineations based on grid assemblages detected more detailed biogeographical transitions, while those based on regional lists showed stronger insularity in biogeographical structure. The phylogenetic regionalization suggested a very similar spatial structure but varied affinities between assemblages in comparison to the taxonomic approach.Main ConclusionsHere, we integrated expert‐validated regional lists, species distribution modelling and a recent phylogeny to tackle Wallacean and Darwinian shortfalls for an important insect group by developing a next‐generation map of biogeographical regionalization for European ants. The results of this study suggest strong constraints from geographical barriers and potential effects of climatic history on ant distributions and evolutionary history, and also provide baseline spatial information for future investigations of regional insect distributions.

An extreme learning machine for unsupervised online anomaly detection in multivariate time series

Unsupervised anomaly detection in time series remains challenging, due to the rare and complex patterns of anomalous data. Previous change point detection methods based on extreme learning machine and mutual information (ELM-MI) are potential solutions for this problem. However, the kernels in these methods are randomly initialized on test data, which imposes a constraint that these methods can only be used for offline inference. Moreover, these methods are limited in utilizing temporal contexts, and require the ensemble of multiple models to improve the robustness. To tackle these problems, we introduce a multivariate ELM-MI framework, and combine it with a dynamic kernel selection method, which performs a hierarchical clustering procedure on unlabeled training data and utilizes the clusters to determine the kernels in ELM-MI. In this way, our method can tackle the unsupervised online detection of various anomalous (e.g., point anomalies and group anomalies) and reduce the computational cost. Extensive experiments on three public datasets and our collection of real-life 4G Long-Term Evolution data demonstrate that the proposed method outperforms state-of-the-art methods in terms of effectiveness and efficiency. For demo, see this link: https://personal.ntu.edu.sg/ezplin/NC-demo.htm.

MULTIVARIATE STATISTICAL ANALYSIS OF REGIONAL ECONOMIC DISPARITIES AT DISTRICT LEVEL IN SERBIA

Starting from the fact that pronounced differences in the level of development of regions within a particular country can have a serious and significant (negative) impact on its socio-political stability, as well as the performance of the national economy as a whole, it is very important to create conditions for ensuring balanced and sustainable regional development. Due to its pronounced multidimensional nature, the analysis of regional economic disparities is a very complex and statistically demanding task. In this paper, a multivariate methodological framework for the classification of districts in Serbia according to the achieved level of economic development, into internally-homogeneous / externally-heterogeneous groups, based primarily on the application of hierarchical agglomerative clustering procedure and examination of interdependencies between five selected relevant economic indicators, is presented. The statistical validity of the obtained "optimal" classification of districts is additionally tested and confirmed with the results of one-factor multivariate analysis of variance. The resulting categorization clearly and unequivocally confirms the presence of pronounced inequalities regarding the achieved level of economic development between NUTS 3 level territorial units in Serbia, and the existence of regional economic polarization, primarily in direction "developed north – undeveloped south".

"CLUSTERING TAX ADMINISTRATIONS IN EUROPEAN UNION MEMBER STATES"

"The European Union Member States use different organizational and functional models of tax administration that could determine better or worse performances. This paper analyzes the way of organization and operation of tax administrations in European Union Member States from the perspective of the 21 variables obtained based on the information made available on the OECD’s Tax Administration Comparative Information Series. Using the hierarchical clustering procedures, tax administrations in the European Union Member States were grouped into clusters. The purpose of this approach was to observe if the respective clusters can be associated with a certain grouping of the tax administrations, made according to their classification, from the point of view concerning the activity efficiency. The efficiency of the activity was evaluated based on 5 indicators developed in the specialty literature. The research showed that the tax administrations in the formed clusters can be found in one of the ranking parts. Therefore, the grouping of tax administrations based on variables that reflect their characteristics can be a useful tool in identifying an organization and functioning model for the tax administration that associated with a certain efficiency level."

A Roadmap towards Breast Cancer Therapies Supported by Explainable Artificial Intelligence

In recent years personalized medicine reached an increasing importance, especially in the design of oncological therapies. In particular, the development of patients’ profiling strategies suggests the possibility of promising rewards. In this work, we present an explainable artificial intelligence (XAI) framework based on an adaptive dimensional reduction which (i) outlines the most important clinical features for oncological patients’ profiling and (ii), based on these features, determines the profile, i.e., the cluster a patient belongs to. For these purposes, we collected a cohort of 267 breast cancer patients. The adopted dimensional reduction method determines the relevant subspace where distances among patients are used by a hierarchical clustering procedure to identify the corresponding optimal categories. Our results demonstrate how the molecular subtype is the most important feature for clustering. Then, we assessed the robustness of current therapies and guidelines; our findings show a striking correspondence between available patients’ profiles determined in an unsupervised way and either molecular subtypes or therapies chosen according to guidelines, which guarantees the interpretability characterizing explainable approaches to machine learning techniques. Accordingly, our work suggests the possibility to design data-driven therapies to emphasize the differences observed among the patients.

Efficient Big Text Data Clustering Algorithms using Hadoop and Spark

Document clustering is a traditional, efficient and yet quite effective, text mining technique when we need to get a better insight of the documents of a collection that could be grouped together. The K-Means algorithm and the Hierarchical Agglomerative Clustering (HAC) algorithm are two of the most known and commonly used clustering algorithms; the former due to its low time cost and the latter due to its accuracy. However, even the use of K-Means in text clustering over large-scale collections can lead to unacceptable time costs. In this paper we first address some of the most valuable approaches for document clustering over such 'big data' (large-scale) collections. We then present two very promising alternatives: (a) a variation of an existing K-Means-based fast clustering technique (known as BigKClustering - BKC) so that it can be applied in document clustering, and (b) a hybrid clustering approach based on a customized version of the Buckshot algorithm, which first applies a hierarchical clustering procedure on a sample of the input dataset and then it uses the results as the initial centers for a K-Means based assignment of the rest of the documents, with very few iterations. We also give highly efficient adaptations of the proposed techniques in the MapReduce model which are then experimentally tested using Apache Hadoop and Spark over a real cluster environment. As it comes out of the experiments, they both lead to acceptable clustering quality as well as to significant time improvements (compared to K-Means - especially the Buckshot-based algorithm), thus constituting very promising alternatives for big document collections.

Bathymetric Particle Filter SLAM Based on Mean Trajectory Map Representation

To obtain independent navigation results for autonomous underwater vehicles (AUVs) and construct high-resolution consistent seabed maps, a particle filter-based bathymetric simultaneous localization and mapping (BSLAM) method with the mean trajectory map representation is proposed. To reduce the computational consumption, particles only keep the current estimated position of the vehicle, while all historical states of the vehicle are stored in the mean trajectory map. Using this set-up, only the weights of the particles which closed to the mean trajectory map are calculated with newly collected bathymetric data. A hierarchical clustering procedure is also discussed to identify invalid loop closures. The performance of the proposed method is validated using both the simulated data and the field data collected from sea trails. The results demonstrate that the proposed method is 50% more accurate and 50% faster than a state-of-the-art particle filter-based BSLAM method, and it has similar accuracy but 30% faster compared with a graph-based BSLAM method.

Governance models for nature-based solutions: Seventeen cases from Germany

Nature-based solutions (NBS) for mitigating climate change are gaining popularity. The number of NBS is increasing, but research gaps still exist at the governance level. The objectives of this paper are (i) to give an overview of the implemented NBS for flood risk management and mitigation in Germany, (ii) to identify governance models that are applied, and (iii) to explore the differences between these models. The results of a hierarchical clustering procedure and a qualitative analysis show that while no one-size-fits-all governance model exists, polycentricism is an important commonality between the projects. The study concludes by highlighting the need for further research on traditional governance model reconversion and paradigm changes. We expect the findings to identify what has worked in the past, as well as what is important for the implementation of NBS for flood risk management in future projects.

Agritourism and Sustainability: What We Can Learn from a Systematic Literature Review

Scholars from different perspectives agree that agritourism can be the right tool to balance the needs of tourists with those of rural communities, offering real opportunities for economic and social development, while mitigating undesirable impacts on the environment. This paper aims to provide a holistic outlook of the different perspectives under which scientific literature deals with the topic of agritourism as a means to support the sustainable development of rural areas. To reach this aim, we performed a systematic review of the scientific literature in order to point out the linkages between agritourism and sustainability. We analyzed papers through a text mining solution based on the Latent Dirichlet Allocation (LDA) technique to point out the main topics around which the scientific literature on agritourism and sustainability has grown. Topics are further categorized in themes by means of an agglomerative hierarchical clustering procedure. Results are further analyzed to highlight the strengths and weaknesses of the current streams of the literature.

Shape analysis of H ii regions – II. Synthetic observations

ABSTRACT The statistical shape analysis method developed for probing the link between physical parameters and morphologies of Galactic H ii regions is applied here to a set of synthetic observations (SOs) of a numerically modelled H ii region. The systematic extraction of H ii region shape, presented in the first paper of this series, allows for a quantifiable confirmation of the accuracy of the numerical simulation, with respect to the real observational counterparts of the resulting SOs. A further aim of this investigation is to determine whether such SOs can be used for direct interpretation of the observational data, in a future supervised classification scheme based upon H ii region shape. The numerical H ii region data were the result of photoionization and radiation pressure feedback of a 34 M⊙ star, in a 1000 M⊙ cloud. The SOs analysed herein comprised four evolutionary snapshots (0.1, 0.2, 0.4, and 0.6 Myr), and multiple viewing projection angles. The shape analysis results provided conclusive evidence of the efficacy of the numerical simulations. When comparing the shapes of the synthetic regions to their observational counterparts, the SOs were grouped in amongst the Galactic H ii regions by the hierarchical clustering procedure. There was also an association between the evolutionary distribution of regions and the respective groups. This suggested that the shape analysis method could be further developed for morphological classification of H ii regions by using a synthetic data training set, with differing initial conditions of well-defined parameters.

Optimized inspection of upstream oil and gas methane emissions using airborne LiDAR surveillance

Methane is a short-lived climate pollutant responsible for approximately 20% of anthropogenic global warming, and reducing methane emissions from the oil and gas (O&G) industry is considered among the most urgent and actionable measures to mitigate climate change. Recent reports suggest a large fraction of upstream O&G methane emissions result from a small number of super-emitter facilities, emphasizing the value of novel methods that inspect O&G facilities with greater frequency than is practical using existing techniques. Here we describe an optimized method wherein O&G facilities are inspected for emissions at high frequency and high sensitivity using active laser (LiDAR) sensors mounted to aircraft. The method relies on a hierarchical clustering and routing procedure to establish optimal routes to be flown by aircraft departing from local airports and equipped with LiDAR methane sensors. Routes are optimized to inspect all well sites subject to emissions regulation in three O&G intensive regions: the Permian basin, the state of Colorado, and the state of Pennsylvania. While some cost estimates require additional field data, these modeling results suggest the optimized inspections can be performed with comparable effectiveness and up to a factor of six lower cost per inspection compared to current detection methods. The cost per inspection required to achieve equivalent emissions reduction depends on factors such as the weather conditions during inspection (which impacts the limit of detection and therefore the inspection frequency required to achieve equivalency) and the well density (which impacts the flying distance), and the advantage of this program over traditional inspection will be reduced under unfavorable conditions. These modeling results suggest that optimized routing may enable frequent inspection of upstream O&G facilities at large scale and potentially lead to a substantial decrease in both oilfield methane emissions and compliance costs borne by industry.

Lignicolous fungal assemblages and relationships with environment in broadleaved and mixed forests from the North-East Region of Romania

Background and aims – Lignicolous fungal assemblages perform numerous functions in forest ecosystems, one of the most important being their capacity to decay wood. As a consequence of their belonging to different ecological niches, the forest ecosystem influences the fungal assemblages in terms of species richness and composition. Methods – In this study we analyzed the main lignicolous macrofungal assemblages in some deciduous and mixed deciduous-coniferous forests in the North-East Region of Romania. We searched to find fungal indicator species for a certain forest type and which are the main drivers and their effects on the composition of the lignicolous macrofungal assemblages. Fungal assemblages were identified using a hierarchical agglomerative clustering procedure, while diagnostic species for each cluster were identified based on the indicator value index. Relationships between fungal composition of plots and environmental variables were performed using detrended and canonical correspondence analyses.Key results – A total of 377 fungal taxa in approximately 4600 records (in 59 plots) were identified. Six distinct clusters of lignicolous fungal assemblages were defined and separated three groups: 1) species-rich lignicolous fungal assemblages in beech forests (1 cluster), 2) well defined fungal assemblages in the mixed broadleaved-coniferous forests (2 clusters), and 3) fungal assemblages typical to oak forests (3 clusters). Ordination methods highlighted the forest type as the most important factor influencing the fungal composition of plots. Forestry Aridity Index, tree diversity and large trees basal area were also important factors for fungal assemblages but with a lower contribution. Conclusion – In the studied region, fungal assemblages changed from oak to beech and to mixed, broadleaved-coniferous forests mainly as a consequence of different tree composition. Climate also shaped fungal composition but to a lesser extent.