Disruptive factors in the evaluation of perceived quality aspects – consideration of the brand influence

Abstract

The implementation of evaluation methods with the aim to gain knowledge of the perceived quality aspects is a challenging task because of versatile disruptive factors on the users’ cognition. Commonly known disruptive factors like the different level of experience in the usage of a system or the publicity of a brand name are considered in research approaches only marginally. This becomes even more important when the evaluated system stays in direct interaction with the evaluator and an increase of external influences can be expected. On the example of a system with strong user interaction, a research approach for an objective evaluation of quality aspects was developed with cordless drills. An experiment has been carried out with 50 craftsmen which are divided into two groups, whereby the first of which will be given original labeled and the other blinded cordless drills for rating the application quality. Evaluation was divided into a preliminary test and a practical test phase, in which selected applications were performed. Results for the preliminary evaluation of the cordless drills indicate a significant brand influence with a strong positive effect on well-known (p<.001, r=0.492, +30.7%) and a negative effect on unknown brands (p=0.029, r=0.309, -17.2%). Practical usage of the devices leads to a significant change of ratings (well-know brand: p=0.007, r=0.378, +1.9%; unknown brand: p=0.053, r=0.274, +12%). Using paired sample t-test it can be shown that the practical use of cordless drills reduces variances in evaluation and ratings become more precise.Relevance for industry: Given approaches can be used in industry to conduct field studies with real users and benchmark systems to identify new development goals (quality aspects) for user-optimized products that are not affected by disruptive factors such as the brand name.