Multidimensional Rating Research Articles

Slovenian Version of the Drug-Induced Extrapyramidal Symptoms Scale: Evaluation of Interrater and Test-Retest Reliability.

The Drug-Induced Extrapyramidal Symptom Scale (DIEPSS) is a multidimensional rating scale for the assessment of drug-induced extrapyramidal symptoms (EPS), developed in 1994. It is suitable for evaluating EPS considering the degree of influence EPS has on daily activities and the subjective distress that it causes. This study to evaluate the interrater and test-retest reliability of the DIEPSS Slovenian version conducted at the University Medical Center Maribor in Slovenia in November 2018. Six raters performed the interrater assessment of 135 DIEPSS video clips with recordings of patients with EPS. A second assessment was then performed by 2 raters to evaluate the test-retest reliability, which was high (interclass correlation coefficients from 0.743 to 0.936). The results for the Slovenian language version of the DIEPSS show high interrater and test-retest reliability, with high concordance rates for all evaluated items (interclass correlation coefficient > 0.8).

Is the total score of the Hamilton Depression Rating Scale affected by side effects of SSRIs and SNRIs?

Abstract This talk will focus on the pitfalls of using multidimensional rating scales to measure the severity of depression – with particular emphasis on the Hamilton Depression Rating Scale (HDRS-17). First, the history behind the development of the HDRS-17 will be briefly covered. Second, it will be argued that the HDRS-17 measures symptoms that overlap with common antidepressant side effects (gastrointestinal dysfunction, sexual dysfunction, somatic anxiety and sleep disturbances), making it possible that side effects of antidepressant treatment are erroneously rated as symptoms of depression. The rest of the talk will focus on the results of a recent study (1) in which we used individual-level data from antidepressant treatment trials to assess whether side effect ratings co-vary with HDRS-17 ratings. Specifically, data from all HDRS-17-rated, industry-sponsored pre- and post-marketing trials (n = 4647) comparing the serotonin and noradrenaline reuptake inhibitor, duloxetine, to placebo and/or to a selective serotonin reuptake inhibitor were pooled. Severity was assessed for side effects related to sleep, somatic anxiety, gastrointestinal function, and sexual dysfunction. Analysis of covariance was used to assess the relation between these side effects and ratings of relevant HDRS-17-derived outcome parameters. Side effects related to sleep, somatic anxiety and sexual dysfunction significantly and exclusively associated with higher scores on HDRS-17 items measuring the corresponding domains. Side effects related to gastrointestinal function associated with higher HDRS-17 item scores on all assessed domains. Treatment outcome was significantly related to side effect severity when assessed using HDRS-17-sum (beta 0.32 (0.074), p < 0.001), but not when the HDRS-6-sum-score (beta 0.035 (0.043), p = 0.415) or the depressed mood item (beta 0.007 (0.012), p = .527) were used as effect parameters. That some HDRS-17 items co-vary with common antidepressant side effects likely leads to an underestimation of antidepressant efficacy. Finally, based on data from a recent study (2), it will be argued that the Montgomery-Åsberg Depression Rating Scale is biased in the same direction as the HDRS-17 (underestimates antidepressant efficacy), albeit to a lesser extent.

The value of rating diversity within multidimensional rating system: Evidence from hotel booking platform

An increasing number of hotel booking platforms and review websites deploy multidimensional rating systems to encourage users to provide additional evaluations of a product/service besides the overall rating. However, providing the itemized ratings of a product/service requires extra effort which makes users tend to post identical ratings (default setting). According to the information transfer theory, we postulate that rating diversity enhances review usefulness within the multidimensional rating system and; based on the loss aversion phenomenon, we also posit that the positive effect of rating diversity on review usefulness should be greater among negative reviews. Using a sample of 1,720,429 hotel reviews, this study reveals that while rating diversity enhances review usefulness, its effect varies across the valence of each review; specifically, the positive effect of rating diversity mainly functions among negative reviews. These findings yield direct implications for hotel booking platforms or review websites that deploy multidimensional rating systems.

Pythagorean Fuzzy Bonferroni Mean with Weighted Interaction Operator and Its Application in Fusion of Online Multidimensional Ratings

Aiming at the aggregation problem of online multi-attribute interactive ratings, a decision-making method based on Pythagorean Fuzzy Bonferroni mean with weighted interaction (PFWIBM) operator is developed by mining the advantages of ratings driven decision-making and the quantitative advantages of Pythagorean fuzzy sets on linguistic ratings. First, the shortcomings of existing operators in criterion interactive representation are analyzed, and the improved Bonferroni mean with weighted interaction and its dual operator are defined, which are used to aggregate the membership information and non-membership information under the Pythagorean fuzzy environment, respectively, and then the PFWIBM operator is constructed. Second, a generation method of interaction coefficients embedded in the PFWIBM operator is designed by combining expert knowledge and user ratings to overcome the limitations of the method of relying on subjective setting parameter values. Third, based on the proposed operator and the transformation relationship between Pythagorean fuzzy sets and linguistic ratings, an online multi-dimensional rating aggregation decision-making approach for solving product raking problems is developed. Finally, the proposed method is applied to a passenger car ranking example to show the feasibility of the method, and through some comparative analysis, the advantages of the proposed operator are clarified, and the influence of operator parameters on the decision-making results is analyzed.

An incentive mechanism in expert-decision-based crowdsensing networks

With the rapid development of wireless communication and smart devices, crowdsensing applications became popular due to their flexibility to deploy and low cost use. Incentive mechanism is one of the most important research contents in crowdsensing, about crowdsensing incentive mechanism, most existing data quality evaluation methods measure the contributions of users only in terms of data quality, and ignore to measure the sensing cost of users. This leads to the problems of different quality evaluation standards, difficult to measure the data quality and difficult to give a reasonable and effective evaluation to complex problems. However, expert-decision can effectively solve these problems and give high-quality evaluation decision for complex and numerous data results. In this paper, aiming at the shortcomings of existing research, we propose an expert-decision-based crowdsensing framework and gives the multidimensional rating for incentive mechanism based on user cost and data quality (MRAI-UCDQ), which consists of user cost evaluation model, data quality evaluation model, contribution quantification and reward distribution by analysing user sensing cost data and collected sensing data (comprehensive evaluation with quantitative and qualitative analysis). Finally, through nearly 30 days of real experiments, 159 volunteers were recruited and 7000 pieces of sensory data were collected. The result shows the MRAI-UCDQ improves the evaluation performance of data quality and stimulates the user’s perceived participation.

When Paying for Reviews Pays Off: The Case of Performance-Contingent Monetary Rewards

Product review platforms in online marketplaces differ with respect to the granularity of product quality information they provide. While some platforms provide a single overall rating for product quality (also referred to as the single-dimensional rating scheme), others provide a separate rating for each individual quality attribute (also referred to as the multidimensional rating scheme). The multidimensional rating scheme is superior to the single-dimensional rating scheme, ceteris paribus, in reducing consumers’ uncertainty about product quality and value. However, we show that, when sellers respond to product ratings by adjusting their prices, compared to the single-dimensional rating scheme, the multidimensional rating scheme does not always benefit consumers, nor does it necessarily benefit sellers or society. The uncertainty associated with quality attribute rating and the extent of differentiation between competing products determines whether a finer-grained multidimensional rating scheme is superior to a coarser-grained single-dimensional rating scheme from the consumer, seller, and social planner perspectives. The main driver of the results is that more (less) granular and less (more) uncertain information exposes (hides) underlying differentiation, or a lack thereof, between competing products, which, in turn, alters upstream price competition in the presence of heterogeneous consumer preferences. The results demonstrate that focusing on the information transfer aspect of rating schemes provides only a partial understanding of the true impacts of rating schemes.

How do Aspects of Chain Restaurants Affect the Overall Rating: Trip-Advisor Multi-dimensional Rating System Analysis

How do Aspects of Chain Restaurants Affect the Overall Rating: Trip-Advisor Multi-dimensional Rating System Analysis

Effects of Binary Vectors Similarities on the Accuracy of Multi-Criteria Collaborative Filtering

Recommender systems offer tailored recommendations by employing various algorithms, and collaborative filtering is one of the well-known and commonly used of those. A traditional collaborative filtering system allows users to rate on a single criterion. However, a single criterion may be insufficient to indicate preferences in domains such as restaurants, movies, or tourism. Multi-criteria collaborative filtering provides a multi-dimensional rating option. In similarity-based multi-criteria collaborative filtering schemes, existing similarity methods utilize co-users or co-items regardless of how many there are. However, a high correlation with a few co-ratings does not always provide a reliable neighborhood. Therefore, it is very common, in both single- and multi-criteria collaborative filtering, to weight similarities with functions utilizing the number of co-ratings. Since multi-criteria collaborative filtering is yet growing, it lacks a comprehensive view of the effects of similarity weighting. This work studies multi-criteria collaborative filtering and the literature of binary vector similarities, which are frequently used for weighting, by giving a related taxonomy and conducts extensive experiments to analyze the effects of weighting similarities on item- and user-based multi-criteria collaborative filtering. Experimental findings suggest that prediction accuracy of item-based multi-criteria collaborative filtering can be boosted by especially binary vector similarity measures which do not consider mutual absences.

Online Product Reviews: Is a Finer-Grained Rating Scheme Superior to a Coarser One?

Product review platforms in online marketplaces differ with respect to the granularity of product quality information they provide. While some platforms provide a single overall rating for product quality (also referred to as the single-dimensional rating scheme), others provide a separate rating for each individual quality attribute (also referred to as the multidimensional rating scheme). The multidimensional rating scheme is superior to the single-dimensional rating scheme, ceteris paribus, in reducing consumers’ uncertainty about product quality and value. However, we show that, when sellers respond to product ratings by adjusting their prices, compared to the single-dimensional rating scheme, the multidimensional rating scheme does not always benefit consumers, nor does it necessarily benefit sellers or society. The uncertainty associated with quality attribute rating and the extent of differentiation between competing products determines whether a finer-grained multidimensional rating scheme is superior to a coarser-grained single-dimensional rating scheme from the consumer, seller, and social planner perspectives. The main driver of the results is that more (less) granular and less (more) uncertain information exposes (hides) underlying differentiation, or a lack thereof, between competing products, which, in turn, alters upstream price competition in the presence of heterogeneous consumer preferences. The results demonstrate that focusing on the information transfer aspect of rating schemes provides only a partial understanding of the true impacts of rating schemes.

Validation of the Danish versions of the Bristol Rheumatoid Arthritis Fatigue Multi-Dimensional Questionnaires (BRAFs)

Objective: This study aimed to validate the Danish versions of the Bristol Rheumatoid Arthritis Fatigue Multi-Dimensional Questionnaire (BRAF-MDQ) and BRAF Numerical Rating Scale version 2 (NRSv2). Method: We tested face and content validity, internal consistency, criterion validity, construct validity, and reproducibility for the BRAF-MDQ, and face and criterion validity and reproducibility for the BRAF-NRS. Results: In all, 224/236 patients (95%) completed the questionnaires [70% female, mean ± sd age 59 ± 13.04 years, disease duration 11.2 ± 9.49 years, Health Assessment Questionnaire (HAQ) 0.724 ± 0.70, and 28-joint Disease Activity Score–C-reactive protein 2.55 ± 1.24]. The unidimensionality for the physical and cognitive fatigue subscales was confirmed, whereas the living with fatigue and emotional fatigue subscales were not unidimensional. Cronbach’s α was 0.94 for the BRAF-MDQ total and 0.78–0.92 for the four subscales. The correlations between BRAF-MDQ and various measures were: 36-item Short Form Health Survey (SF-36) vitality subscale, 0.75; Hospital Anxiety and Depression Scale (HADS) anxiety subscale, 0.65; HADS depression subscale, 0.62; visual analogue scale (VAS) pain, 0.62; VAS global, 0.73; and HAQ, 0.62. The intraclass correlation coefficient for agreement was 0.995. A Bland–Altman plot showed a mean ± sd difference of −1.9 ± 3.62 for BRAF-MDQ. Correlation coefficients between the BRAF-NRSv2 subscales and other subscales were: BRAF-MDQ subscales, 0.57–0.93; SF-36 vitality subscale, 0.54–0.68; and VAS fatigue, 0.66–0.82. Conclusions: The Danish BRAFs are considered valid and reliable for use among Danish patients with rheumatoid arthritis, despite the subscales living with fatigue and emotional fatigue not being unidimensional, as they are in the original version.

When the Stars Shine Too Bright: The Influence of Multidimensional Ratings on Online Consumer Ratings

Scholars generally assume that consumer ratings reflect consumer satisfaction, but ratings can be influenced by the design of the rating system. We examine two rating designs—single-dimensional rating systems, which elicit overall ratings only, and multidimensional (MD) rating systems, which elicit both dimensional and overall ratings—and how they impact overall ratings. Drawing on the accessibility–diagnosticity framework, we argue that dimensional ratings in MD systems influence overall ratings based on how the dimensions have been rated. We support this explanation with seven experiments. Our results suggest that across various experimental settings, rating objects, dimensions, and numbers of dimensions, overall ratings are systematically influenced by the design of the rating system.This paper was accepted by Chris Forman, information systems.

Evolutive preference analysis with online consumer ratings

The rapid growth of online rating information enables firms to learn and monitor consumer preferences over their products. Based on multidimensional rating information systems, we propose a novel class of Evolutive Preference Analysis (EPA) methods to handle the dynamic online ratings with arbitrary rating distribution, which takes all the historical ratings into consideration and delivers a comprehensive ranking evolution. As a new tool to analyze real-time consumer preferences, the EPA class includes the EPA that emphasizes the overall preferences of consumers, and the EPAt that emphasizes the recent preferences that generally provide up-to-date information. Both of them include four indices (the expected priority vector, expected rank, confidence factor, and index of rank probability) to reveal consumer preferences over rated attributes of products along time. The evolution trends of these indices help firms verify whether their advertised products match the preferences of target consumers to improve their products and marketing strategies. Finally, the practical applicability of EPA class is corroborated by several numerical examples and one real-world application on smartphone online ratings, which demonstrates the effectiveness of the proposed EPA class on streaming rating evolution.

The Efficacy of Collaborative Referencing Intervention in Chronic Aphasia: A Mixed-Methods Study.

Purpose The feasibility of a collaborative referencing intervention (CRI) for adults with chronic aphasia has been documented in two descriptive case studies (Devanga, 2014; Hengst et al., 2010, 2008). The current Phase II mixed-methods treatment study replicates the CRI with four additional participants (using interpretive research) and investigates how it impacts a traditional measure, participants' confrontational naming abilities, outside of game play (using multiple-probe single-case experimental design). Method Four adults with chronic aphasia participated in the study composed of (a) three preparatory sessions, (b) five baseline sessions, (c) 15 CRI sessions with five treatment probes, and (d) six maintenance sessions. A collaborative confrontation naming (CCN) probe (i.e., dependent variable) was administered in each baseline, treatment probe, and maintenance session. Each CRI session (i.e., independent variable) consisted of a photo-matching game with participant and clinician partner taking alternative turns identifying and matching personally relevant treatment cards. CCN probes were scored using a multidimensional rating scale. Fidelity and social validity were also assessed. Results Replication of the CRI showed successful and consistent referential learning in all four participant pairs. The multiple-probe analysis of CCN revealed a positive treatment effect on naming in three participants indicating that the CRI was efficacious. High fidelity was maintained throughout the study. Social validity interviews revealed positive outcomes and significant impacts of treatment on the participants' lives. Conclusion The CRI demonstrates strong clinical implications for adults with chronic aphasia. Future research exploring the treatment effectiveness and the implementation to a variety of clinical settings is warranted.

How fluent? Part B. Underlying contributors to continuous measures of fluency in aphasia

ABSTRACT Background While persons with aphasia (PwA) are often dichotomised as fluent or nonfluent, agreement that fluency is not an all-or-nothing construct has led to the use of continuous variables as a way to quantify fluency, such as multi-dimensional rating scales, speech rate, and utterance length. Though these measures are often used in research, they provide little information about the underlying fluency deficit. Aim The aim of the study was to identify how well commonly used continuous measures of fluency capture variability in spontaneous speech variables at lexical, grammatical, and speech production levels. Methods & Procedures Speech samples of 254 English-speaking PwA from the AphasiaBank database were analyzed to examine the distributions of four continuous measures of fluency: the WAB-R fluency scale, utterance length, retracing, and speech rate. Linear regression was used to identify spontaneous speech predictors contributing to each fluency outcome measure. Outcomes & Results All the outcome measures reflected the influence of multiple underlying dimensions, although the predictors varied. The WAB-R fluency scale, speech rate, and retracing were influenced by measures of grammatical competence, lexical retrieval, and speech production, whereas utterance length was influenced only by measures of grammatical competence and lexical retrieval. The strongest predictor of WAB-R fluency was aphasia severity, whereas the strongest predictor for all other fluency proxy measures was grammatical complexity. Conclusions Continuous measures allow a variety of ways to objectively quantify speech fluency; however, they reflect superficial manifestations of fluency that may be affected by multiple underlying deficits. Furthermore, the deficits underlying different measures vary, which may reduce the reliability of fluency diagnoses. Capturing these differences at the individual level is critical to accurate diagnosis and appropriately targeted therapy.

Gender biases in impressions from faces: Empirical studies and computational models.

Trustworthiness and dominance impressions summarize trait judgments from faces. Judgments on these key traits are negatively correlated to each other in impressions of female faces, implying less differentiated impressions of female faces. Here we test whether this is true across many trait judgments and whether less differentiated impressions of female faces originate in different facial information used for male and female impressions or different evaluation of the same information. Using multidimensional rating datasets and data-driven modeling, we show that (a) impressions of women are less differentiated and more valence-laden than impressions of men and find that (b) these impressions are based on similar visual information across face genders. Female face impressions were more highly intercorrelated and were better explained by valence (Study 1). These intercorrelations were higher when raters more strongly endorsed gender stereotypes. Despite the gender difference, male and female impression models-derived from separate trustworthiness and dominance ratings of male and female faces-were similar to each other (Study 2). Further, both male and female models could manipulate impressions of faces of both genders (Study 3). The results highlight the high-level, evaluative effect of face gender in impression formation-women are judged negatively to the extent their looks do not conform to expectations, not because people use different facial information across genders but because people evaluate the information differently across genders. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Helpfulness and economic impact of multidimensional rating systems: Perspective of functional and hedonic characteristics

AbstractNowadays, the multidimensional rating systems are increasingly popular in practice to help consumers evaluate multiattribute products or services. This study aims to examine the helpfulness and economic impact of multidimensional rating systems and the moderating effect of product type (ordinary, premium) from the perspective of product line. To identify the different effects of different product characteristics on consumers' behavior, multidimensional ratings and textual reviews are classified into two different sets according to the product attributes (functional, hedonic). Besides, the effects of review valence are considered simultaneously. Econometric analysis and text mining are conducted on the real data from autohome.com, a leading Chinese automobile rating platform. Results show that (a) in terms of perceived helpfulness, reviews with positive functional ratings or with negative hedonic ratings are perceived more helpful for ordinary product, whereas reviews with positive hedonic ratings are perceived more helpful for premium product. (b) Regarding product sales, the impacts of functional and hedonic ratings on product sales are inverted U‐shape. Furthermore, for ordinary product, variance of functional ratings has a negative effect on product sales, whereas hedonic ratings variance has an opposite impact. Surprisingly, ordinary product consumers tend to pay more attention to the negative hedonic reviews, whereas premium product consumers concentrate more on the negative functional reviews. In summary, various influences of multidimensional reviews and ratings across product type on review helpfulness and product sales are deeply explored in this study, suggesting that firms' online marketing strategies should be contingent on product type and characteristics.

A Social Desirability Item Response Theory Model: Retrieve-Deceive-Transfer.

In this study, a new item response theory model is developed to account for situations in which respondents overreport or underreport their actual opinions on a positive or negative issue. Such behavior is supposed to be a result of deception and transfer mechanisms. In the proposed model, this behavior is simulated by incorporating a deception term into a multidimensional rating scale model, followed by multiplication by a transfer term, with the two operations performed by an indicator function and a transition matrix separately. The proposed model is presented in a Bayesian framework approximated by Markov chain Monte Carlo algorithms. Through a series of simulations, the parameters of the proposed model are recovered accurately. The methodology is also implemented within an online experimental study to demonstrate the methodology's application.

Facial expression processing is not affected by Parkinson’s disease itself, but by age related factors

Although the degree of facial movement decrease in patients with Parkinson’s disease (PD), the question whether facial expression processing may be impaired in PD so far has yielded equivocal results. Recent studies have focused on testing expression processing in recognition tasks with static images, not with dynamic conditions. Both conversational, emotional and dynamic facial expressions in three groups were tested. PD patients (n = 20), age- and education-matched healthy controls (n = 20), and young adult healthy controls (n = 20) were recruited. This setup allows us to address both effects of PD and age-related differences. We employed a multi-dimensional rating task for all groups in which 12 rating dimensions were used to assess evaluative processing of 27 expression videos from 6 different actors. We found that ratings overall were consistent across groups with several rating dimensions having a strong correlation with the expressions’ motion energy content as measured by optic flow analysis. Most importantly, we found that the PD group did not differ in any rating dimension from the older healthy control group, indicating highly similar evaluation processing. Both older groups, however, showed significant differences for several rating scales in comparison with the younger control group. Older participants rated negative expressions compared to the younger participants as more positive, but also as less natural, persuasive, empathic, and sincere. We interpret these in the context of positivity effect and in-group processing advantages. Overall, our findings do not support processing deficits due to PD, but rather point to age-related differences in facial expression processing.

Drug-Induced Extrapyramidal Symptoms Scale of the Norwegian version: inter-rater and test–retest reliability

Background: The Drug-Induced Extrapyramidal Symptoms Scale (DIEPSS) is a multidimensional rating scale designed for the fast, easy and reliable assessment of extrapyramidal symptoms (EPSs) induced by antipsychotics.Aim: The aim of this study was to validate the level of inter-rater and test–retest reliability of the Norwegian translation of this scale.Methods: A total of 125 video clips showing a variety of or no signs of EPSs were used in the present study. The participants recorded were Japanese psychiatric patients receiving first- and/or second-generation antipsychotics. A total of 103 patients (47 males and 56 females), diagnosed with schizophrenia (n = 68) or mood disorders (n = 35) appeared in the video clips. Their mean age was 48.7 ± 16.3 years (range 18–80) at the time of video recording. Inter-rater agreement was assessed with five raters and test–retest reliability with three.Results: Inter-rater reliability analyses showed interclass correlation coefficients (ICCs) ranging from 0.74 to 0.93 for each individual item. Test–retest reliability analysed independently for each rater ranged from 0.71 to 0.96.Conclusions: Inter-rater and test–retest agreement exhibited satisfactory ICC levels above 0.70. The Norwegian version of the DIEPSS is a reliable instrument for the assessment of drug-induced EPSs.

Multidimensional Small Medium Enterprises Achievement Rating : Improving to Data Warehousing & Data Mining Methods

Multidimensional Small Medium Enterprises Achievement Rating : Improving to Data Warehousing & Data Mining Methods