Proposed Diversity Measure Research Articles

Are We There Yet? Analyzing Progress in the Conversion Funnel Using the Diversity of Searched Products

The conversion funnel is a model describing the stages consumers go through in their journey toward a purchase. This journey often lasts several days to weeks and can include multiple visits to a seller’s website. A large body of literature has focused on using observable search patterns to identify consumers’ hidden purchasing stages and to estimate their likelihood of conversion. We propose a novel set of measures to better reveal the consumer’s hidden stage in the funnel. These measures are based on the diversity of the searches that a customer engages in while browsing an e-commerce website, and they include not only the number of different products that are searched for, but also measures that rely on unobserved similarities among products, captured in a product network (in which products are assumed to be “similar” if they are frequently co-searched). We operationalize and evaluate our proposed measures using a large-scale dataset from a medium-sized tourism website used for comparing and booking flights. We estimate a hidden Markov model to show that our proposed diversity measures are associated with progress in the funnel and consumers’ conversion likelihood. Specifically, we show that consumers go through different distinguishable stages (states) in their journey, characterized by different values of our proposed diversity measures. To demonstrate the managerial and business implications of our theory, we show that incorporating search-diversity measures into a baseline prediction model significantly improves the model’s performance in predicting purchase likelihood and churn.

New Diversity Measures Based on the Coverage and Similarity of the Classification

In supervised pattern classification, it often happens that a single individual classifier is not able to meet the requirements of the problem. This is the main reason that leads to the successful use of systems composed of several classifiers (classifier ensembles) looking to obtain better results than a single classifier. The selection of the classifiers to be used is difficult due to its great variety and to the presence of necessary conditions such as the diversity between them to obtain the best possible results. Specifically, the diversity among the classifiers continues being a crucial factor in these systems and the way of measuring it in an effective way is still an open field. There are different measures in literature that help to decide if a group of classifiers is diverse. Some methods explicitly use these measures to obtain more effective classifier ensembles. The obtained results show it is possible to find combinations of classifiers that assure a superior accuracy compare to the best individual accuracy. In this paper, we propose two new diversity measures based on the coverage and similarity of the classification. Our objective is to measure the diversity in a different way in search of better results. We show several experiments where the behavior of the proposed diversity measures is analyzed. Also, we present the correlation that exists among the propose measures and other measures, including the classifier ensemble accuracy.

Clustering ensemble selection based on the extended Jaccard measure

Clustering ensemble selection has shown high efficiency in the improvement of the quality of clustering solutions. This technique comprises two important metrics: diversity and quality. It has been empirically proved that ensembles of higher effectiveness can be achieved through taking into consideration the diversity and quality simultaneously. However, the relationships between these two metrics in base clusterings have remained uncertain. This paper suggests a new hierarchical selection algorithm using a diversity/quality measure based on the Jaccard similarity measure. In the proposed algorithm, the selection of the subsets of the clustering partitions is done based on their diversity measures. The proposed diversity measure (in two types of pair-wise diversity and hybrid diversity) is applied to the proposed algorithm. Hypergraph-partitioning algorithm (HGPA), cluster-based similarity partition algorithm (CSPA), and meta-clustering algorithm (MCLA) were used to obtain the consensus solution and cluster ensemble selection results with a hierarchical method. The experimental results on 14 datasets showed that selecting a subset of base clusterings using the proposed algorithm led to more accurate results compared to those of the full ensemble. The effectiveness and robustness of the proposed algorithm were demonstrated in comparison with the full ensemble. The comparative results showed that the proposed method by new diversity measure outperformed the full ensemble.

Deep into the Funnel? Predicting Online Conversion Using Search Diversity

The conversion funnel is a model describing the stages consumers go through in their journey towards a purchase. This journey often lasts several days to weeks and can include multiple visits to a seller’s website. A large body of literature has focused on using observable search patterns to identify consumers’ hidden purchasing stages and to estimate their likelihood of conversion. We propose a novel set of measures to better unveil the consumer’s hidden stage in the funnel. These measures are based on the diversity of the searches that a customer engages in while browsing an e-commerce website, and they include not only the number of different products that are searched for, but also measures that rely on unobserved similarities among products, captured in a product network (in which products are assumed to be “similar” if they are frequently co-searched). We operationalize and evaluate our proposed measures using a large-scale dataset from a medium-sized tourism website used for comparing and booking flights. We estimate a hidden Markov model to show that our proposed diversity measures are associated with progress in the funnel and consumers' conversion likelihood. Specifically, we show that consumers go through different distinguishable stages (states) in their journey, characterized by different values of our proposed diversity measures. To demonstrate the managerial and business implications of our theory, we show that incorporating search-diversity measures into a baseline prediction model improves the model’s performance in predicting purchase likelihood and churn. For example, the purchase likelihood prediction model’s recall, precision and F1-score improved by factors of 5.64, 5.59, and 6.17 respectively.

Diversity Assessment in Many-Objective Optimization.

Maintaining diversity is one important aim of multiobjective optimization. However, diversity for many-objective optimization problems is less straightforward to define than for multiobjective optimization problems. Inspired by measures for biodiversity, we propose a new diversity metric for many-objective optimization, which is an accumulation of the dissimilarity in the population, where an L p -norm-based ( ) distance is adopted to measure the dissimilarity of solutions. Empirical results demonstrate our proposed metric can more accurately assess the diversity of solutions in various situations. We compare the diversity of the solutions obtained by four popular many-objective evolutionary algorithms using the proposed diversity metric on a large number of benchmark problems with two to ten objectives. The behaviors of different diversity maintenance methodologies in those algorithms are discussed in depth based on the experimental results. Finally, we show that the proposed diversity measure can also be employed for enhancing diversity maintenance or reference set generation in many-objective optimization.

Measuring Plan Diversity: Pathologies in Existing Approaches and A New Plan Distance Metric

In this paper we present a plan-plan distance metric based on Kolmogorov(Algorithmic) complexity. Generating diverse sets of plans is useful for task ssuch as probing user preferences and reasoning about vulnerability to cyberattacks. Generating diverse plans, and comparing different diverse planning approaches requires a domain-independent, theoretically motivated definition of the diversity distance between plans. Previously proposed diversity measures are not theoretically motivated, and can provide inconsistent results on the sameplans. We define the diversity of plans in terms of how surprising one plan is givenanother or, its inverse, the conditional information in one plan givenanother. Kolmogorov complexity provides a domain independent theory of conditional information. While Kolmogorov complexity is not computable, a related metric, Normalized Compression Distance (NCD), provides a well-behaved approximation. In this paper we introduce NCD as an alternative diversity metric, and analyze its performance empirically, in comparison with previous diversity measures, showing strengths and weaknesses of each.We also examine the use of different compressor sin NCD. We show how NCD can be used to select a training set for HTN learning,giving an example of the utility of diversity metrics. We conclude withsuggestions for future work on improving, extending, and applying it to serve new applications.

Hierarchical cluster ensemble selection

Clustering ensemble performance is affected by two main factors: diversity and quality. Selection of a subset of available ensemble members based on diversity and quality often leads to a more accurate ensemble solution. However, there is not a certain relationship between diversity and quality in selection of subset of ensemble members. This paper proposes the Hierarchical Cluster Ensemble Selection (HCES) method and diversity measure to explore how diversity and quality affect final results. The HCES uses single-link, average-link, and complete link agglomerative clustering methods for the selection of ensemble members hierarchically. A pair-wise diversity measure from the recent literature and the proposed diversity measure are applied to these agglomerative clustering algorithms. Using the proposed diversity measure in HCES leads to more diverse ensemble members than that of pairwise diversity measure. Cluster-based Similarity Partition Algorithm (CSPA) and Hypergraph-Partitioning Algorithm (HGPA) were employed in HCES method for obtaining the full ensemble and cluster ensemble selection solution. To evaluate the performance of the HCES method, several experiments were conducted on several real data sets and the obtained results were compared to those of full ensembles. The results showed that the HCES method led to a more significant performance improvement compared with full ensembles.

Diversity Maximization Approach for Multiobjective Optimization

One of the most common approaches for multiobjective optimization is to generate the whole or partial efficient frontier and then decide about the preferred solution in a higher-level decision-making process. In this paper, a new method for generating the efficient frontier for multiobjective problems is developed, called the diversity maximization approach (DMA). This approach is capable of solving mixed-integer and combinatorial problems. The DMA finds Pareto optimal solutions by maximizing a proposed diversity measure and guarantees generating the complete set of efficient points. Given a subset of the efficient frontier, DMA finds the next Pareto optimal solution which, combined with the existing ones, yields the most diversified subset of efficient points. This solution is defined as the most diverse solution. In fact, it aims to maximize the distance between the new efficient point and the closest point in the given subset of the efficient frontier. The proposed approach can be applied to any problem that can be solved for the single-objective case. We can use the DMA by solving directly a modified version of the mixed-integer programming (MIP) formulation of the multiobjective problem. In this case, the Pareto optimal solutions are found sequentially in an iterative way. Consequently, as we terminate the procedure before completion, a partial efficient frontier is available. The diversity measure assures that in every stage of the procedure, the partial efficient frontier is well diversified. This partial efficient frontier can be perceived as a filtered set of the complete efficient frontier and can be used by the decision maker in case the complete efficient frontier contains too many points. An additional way of using DMA is by incorporating it in a problem oriented branch-and-bound algorithm. Detailed examples of both approaches are given.