Outcome Feedback Research Articles

Trait sensation seeking is associated with heightened beta-band oscillatory dynamics over left ventrolateral prefrontal cortex during reward expectancy

BackgroundSensation Seeking, the proclivity toward novel and stimulating experiences, is associated with greater left ventrolateral prefrontal cortex (vlPFC) activity during uncertain reward expectancy. Here, we examined relationships between sensation seeking and vlPFC oscillatory dynamics using electroencephalography (EEG). MethodsIn 26 adolescents/young adults (16 female; 22.3 ± 1.7yrs), EEG was measured during uncertain reward expectancy. Event-related spectral perturbations (ERSP) from 15-80 Hz (beta/gamma bands) were compared as a function of uncertain reward expected value and assessed for relationships with feedback-related negativity (FRN) response to outcome feedback and response tendency measures of risk for BD. ResultsEvent-related synchronization (ERS) between 15-25 Hz (beta) over left vlPFC was sensitive to the expected value of uncertain reward (rho=0.46; p = 0.048), and correlated with sensation seeking (r = 0.49, p < 0.01) and feedback-related negativity (FRN), where greater beta ERS was related to larger FRN (r = -0.39, p = 0.047). FRN was also related to behavioral inhibition (r = 0.49, p < 0.01). LimitationsIt is unknown whether results may extrapolate to clinical populations, given the healthy sample used here. Further, although we have confidence that the beta-band signal we measure in this study arises from left prefrontal cortex, we largely infer a left vlPFC source. ConclusionsThese findings highlight the role of left vlPFC in evaluation of immediate rewards. We now provide a link between reward expectancy-related left vlPFC activity and the well-characterized FRN, with a known role in attentive processing. These findings can guide treatment development for mania/hypomania at-risk individuals, including transcranial alternating current stimulation.

The modulation of attentional deployment on regret: an event-related potential study.

Adopting a sequential risk-taking task, this study explored the modulation of attentional deployment on regret. Attentional deployment was manipulated during outcome feedback of the task by highlighting different parts to induce participants to focus on collected gains (GF context) or missed chances (MF context). The control context without attentional deployment manipulation was also set. Behaviorally, compared to the control context, participants felt less regret in the GF context but more regret in the MF context. Event-related potential results showed that the GF context elicited stronger reward positivity and late positive potential (LPP) than the control context. Furthermore, openness (NEO Five-Factor Inventory) negatively predicted the amplitude of LPP in the GF context. Source localization indicated that the superior frontal gyrus showed stronger activation in the GF context than in the control context during the time window of LPP. These results suggested that focusing on collected gains was an effective way to repress regret and that the LPP component played a key role in this process.

Застосування зворотного зв’язку в автономному навчанні іншомовного спілкування

The article deals with the use of feedback in the autonomous learning of foreign language communication of students in higher education institutions. The authors analyzed the scientific and methodological literature on the research topic, identified the main methodological fundamentals for the feedback application and provided recommendations for the use of feedback in the autonomous learning of foreign language communication of students in higher education institutions. The article identifies the levels of student autonomy, reveals the essence of the outcome and cognitive feedback, as well as the features of internal and external feedback. According to the developed abilities, the levels of student’s autonomy can be the following: a high / advanced level, a moderate / intermediate level, a low / beginning level. It is cleared out that the most frequent outcome feedback gives learners little advice on their own self-study. Scientists often compare cognitive feedback with outcome feedback. The latter also recommends some ways of dealing with difficulties. It is stated that the internal feedback is invoked by students’ monitoring of themselves and links their past outcomes with the expected. Simultaneously, students get external feedback from numerous sources. It can be spontaneous or conscious and intended. Feedback has been found to be a prerequisite for effective foreign language learning for students with different levels of autonomy.

Will We Trust What We Don’t Understand? Impact of Model Interpretability and Outcome Feedback on Trust in AI

Despite AI's superhuman performance in a variety of domains, humans are often unwilling to adopt AI systems. The lack of interpretability inherent in many modern AI techniques is believed to be hurting their adoption, as users may not trust systems whose decision processes they do not understand. We investigate this proposition with a novel experiment in which we use an interactive prediction task to analyze the impact of interpretability and outcome feedback on trust in AI and on human performance in AI-assisted prediction tasks. We find that interpretability led to no robust improvements in trust, while outcome feedback had a significantly greater and more reliable effect. However, both factors had modest effects on participants' task performance. Our findings suggest that (1) factors receiving significant attention, such as interpretability, may be less effective at increasing trust than factors like outcome feedback, and (2) augmenting human performance via AI systems may not be a simple matter of increasing trust in AI, as increased trust is not always associated with equally sizable improvements in performance. These findings invite the research community to focus not only on methods for generating interpretations but also on techniques for ensuring that interpretations impact trust and performance in practice.

A Boo is Louder Than a Cheer: How Rejection and Feedback Type Influence Misreporting

Our study investigates how idea rejection (rejection vs acceptance) and feedback type (outcome feedback vs explanatory feedback) affect employees’ misreporting of private information. Drawing on research in social psychology, we predict that employees will perceive rejections to be an unkind act by managers. Consequently, we expect employees to behave reciprocal and punish the managers by misreporting their private information to a greater extent. Moreover, we predict that the link between rejection (acceptance) and employees’ perceived kindness is dependent on the feedback type managers use. Specifically, when employees’ ideas are rejected, providing explanatory feedback will increase their kindness perceptions, thus, reducing employees’ misreporting. When employees’ ideas are accepted, however, providing explanatory feedback will not affect their misreporting because employees will perceive the acceptance already to be relatively kind. Our findings from a laboratory experiment support our predictions as well as the underlying theoretical processes. The results of our study contribute for one thing to theory by identifying rejection as an important factor that influences employees’ misreporting and for another to practice by supporting managers in designing their feedback processes efficiently.

Implementing evidence-based assessment and selection in organizations: A review and an agenda for future research

In personnel- and educational selection, a substantial gap exists between research and practice, since evidence-based assessment instruments and decision-making procedures are underutilized. We provide an overview of studies that investigated interventions to encourage the use of evidence-based assessment methods, or factors related to their use. The most promising studies were grounded in self-determination theory. Training and autonomy in the design of evidence-based assessment methods were positively related to their use, while negative stakeholder perceptions decreased practitioners’ intentions to use evidence-based assessment methods. Use of evidence-based decision-making procedures was positively related to access to such procedures, information to use it, and autonomy over the procedure, but negatively related to receiving outcome feedback. A review of the professional selection literature showed that the implementation of evidence-based assessment was hardly discussed. We conclude with an agenda for future research on encouraging evidence-based assessment practice.

Investigating Indonesian Senior High School Students’ Implementation of Self-Regulated Learning towards the English Subject

English is considered as a foreign language for Indonesian speakers. It also becomes the first obligatory foreign language to be taught in Indonesian Secondary Schools. However, English is rarely used as the medium of communication in the daily life context. Consequently, learners encounter difficulties in uttering sentences and creating written expressions. Therefore, they need to be self-regulated and strategic to acquire English proficiency. The researchers argued that applying self-regulated learning (SRL) by both students and teachers could enhance students’ language mastery. This paper aimed to investigate whether or not self-regulated learning towards the English subject of Grade X and XI students was implemented in one Senior High School in Yogyakarta, Indonesia. It adopted a mixed method approach by distributing questionnaire to 60 students, and conducting a semi-structured interview to one English teacher. One English teacher was chosen as a convenience sample in this study. Based on the finding, the students depict positive results in responding four strategies represented Self-Regulated Learning, including: 1) Planning, 2) Monitoring, 3) Controlling, and 4) Reflecting. Meanwhile, the researcher classified the findings of the teacher’s interview into four major points. The points were: 1) Modelling, 2) Encouraging, 3) Strategic Teaching Method, 4) Outcome Checking and Feedback. From the findings, it could be concluded that both the students and the teacher implemented some aspects of self-regulated learning; even though the process should be conducted more systematically and consistently. Keywords: English learning strategies, self-regulated learning, autonomous learning, English subject

Dynamic overconfidence: a growth curve and cross lagged analysis of accuracy, confidence, overestimation and their relations

Research has paid little attention to how overconfidence evolves over time. We examined how task experience (experience within a task using a sequence of items) and outcome feedback affected accuracy, confidence and overconfidence in experiments over several trials. We conducted five studies involving 614 participants and used growth curve modelling and cross-lagged analyses. Findings revealed that mere task experience (without feedback) reduced overestimation linearly. Task experience coupled with feedback reduced overconfidence quadratically; the decreasing rate was initially strong but faded away over time. The decrease in overestimation was explained due to accuracy increasing at a faster rate than confidence did. Accuracy had lagged effects on confidence; a correct estimate led to more confidence in a subsequent estimate. We also found some evidence indicating that confidence had a negative lagged influence on accuracy. This dynamic influence between accuracy and confidence is a unique finding in the overconfidence literature.

Innovative “RK Walker”: Transdisciplinary Multiphasic Observational Experimental Trial

Activities of Daily Living (ADLs) involves several physical activities like walking, climbing, etc. However,such ADLs may get compromised because of disorders where assistive devices play a key role for functionalindependence. Walker is a device used by people with locomotor disabilities, where stair climbing remainschallenge. Objectives: To understand the process of making an innovative walker for stair climbing, totest and re-test its applicability with healthy adults and patients with locomotor disabilities, to dedicate theoutcome of research work to improve quality of care and life. Methodology: Multiphasic ObservationalExperimental Trial, Phase I: Identify the need of change in traditional walker by survey, Phase II: Designinga Walker in AutoCAD software and hardware (RK walker), Phase III: To assess the efficacy of “RK Walker”compared to Traditional Walker in healthy adults, Phase IV: With reference to data analysis from healthyadults, Traditional standard Walker and “RK Walker” was compared for walking (TUG) and “RK Walker”for Stair Climbing (SCT) on patients with partially and non-weight bearing gait due to spinal / lower limbmusculoskeletal dysfunction. Result: Outcome measures and feedback reports show significant impact withRK Walker as compared to traditional walker especially in stair climbing. Conclusion: This multiphasicobservational experimental trial and novel concept extend the functional ability for stair climbing for patientswith disability.

Attentional deployment training impacts neural responses to subsequent regret

As a common negative emotion in daily life, regret is harmful to mental health and even inclined to induce mental illness. Therefore, to modulate and reduce regret is of wide concern. The current fMRI study aimed to investigate the modulation of attentional deployment training on subsequent regret by using a sequential risk taking task. On each trial of the task, participants were asked to open a series of boxes consecutively and decided when to stop. Each box contained a reward, except for one containing a devil to zero one's gain in the trial. After participants stopped, both collected gains and missed chances were revealed. The study consisted of training and testing stages. At training stage, attentional deployment manipulation was carried out during the outcome feedback of the task, i.e. participants were induced to focus on gains or missed chances by highlighting different parts of the outcome. In the testing stage followed, participants completed the task with no attentional deployment manipulation. Behaviorally, after a training stage focusing on gains, participants felt less regret when encountered with worse outcome at testing stage. At the neural level, more activities of reward related brain regions such as right putamen and cognitive control related brain regions such as dmPFC, SFG and MFG were observed at testing stage followed by a training stage focusing on gains. The current study highlighted that attentional deployment training could modulate the subsequent regret effectively, and that dmPFC, SFG and MFG played a key role in this process.

Tonically Active Neurons in the Monkey Dorsal Striatum Signal Outcome Feedback during Trial-and-error Search Behavior

An animal’s choice behavior is shaped by the outcome feedback from selected actions in a trial-and-error approach. Tonically active neurons (TANs), presumed cholinergic interneurons in the striatum, are thought to be involved in the learning and performance of reward-directed behaviors, but it remains unclear how TANs are involved in shaping reward-directed choice behaviors based on the outcome feedback. To this end, we recorded activity of TANs from the dorsal striatum of two macaque monkeys (Macaca fuscata; 1 male, 1 female) while they performed a multi-step choice task to obtain multiple rewards. In this task, the monkeys first searched for a rewarding target from among three alternatives in a trial-and-error manner and then earned additional rewards by repeatedly choosing the rewarded target. We found that a considerable proportion of TANs selectively responded to either the reward or the no-reward outcome feedback during the trial-and-error search, but these feedback responses were not observed during repeat trials. Moreover, the feedback responses of TANs were similarly observed in any search trials, without distinctions regarding the predicted probability of rewards and the location of chosen targets. Unambiguously, TANs detected reward and no-reward feedback specifically when the monkeys performed trial-and-error searches, in which the monkeys were learning the value of the targets and adjusting their subsequent choice behavior based on the reward and no-reward feedback. These results suggest that striatal cholinergic interneurons signal outcome feedback specifically during search behavior, in circumstances where the choice outcomes cannot be predicted with certainty by the animals.

Task satisfaction: the effect of relative performance information versus outcome feedback

PurposeIn this empirical study, the primary aim is to examine whether the type of feedback provided, relative performance information (RPI) vs outcome, affects individual's task satisfaction in a context without financial incentives. A secondary objective is to explore whether differences in individuals' task satisfaction were associated with their performance level.Design/methodology/approachParticipants completed a mundane, effort-based task in a 1 × 2 between-subjects experimental design where the type of feedback was manipulated at two levels (RPI vs outcome).FindingsThe results revealed a positive link between providing RPI feedback to individuals and their self-reported task satisfaction compared to individuals provided with outcome feedback. We find that individuals' task satisfaction is not associated with their task performance, supporting our prediction that the level of knowledge of results affects individuals' task satisfaction.Research limitations/implicationsThe experimental task used in this study was mundane and effort intensive. Consequently, future research may be needed to examine whether the results generalize to more creative, less effort-intensive tasks. This study also utilized student participants as a proxy for employees, which is appropriate for the task, but may not generalize to organizational settings requiring specialized knowledge or task experience.Practical implicationsThe findings suggest that organizations may find providing employees with RPI to be a relatively low-cost, non-financial incentive for improving employee task satisfaction, a construct documented to reduce employee turnover and absenteeism.Originality/valueWhile prior research focuses on the effects of providing RPI on individuals' performance and effort, this study extends prior research to individuals' task satisfaction, an affective construct, illustrating that RPI is multi-dimensional. Our results also have implications for theory. We extend the feedback proposition of the widely applied Job Characteristics Model (JCM) by illustrating the type of feedback provided to individuals has task satisfaction effects beyond those associated with the mere presence of feedback.

Choices in a key press decision-making task are more optimal after gaining both aiming and reward experience.

When presented with two different target-penalty configurations of similar maximum expected gain (MEG), participants prefer aiming to configurations with more advantageous spatial, rather than more advantageous gain parameters-perhaps due to the motor system's inherent prioritisation of spatial information during movements with high accuracy demands such as aiming. To test this hypothesis, participants in the present studies chose between target-penalty configurations via key presses to reduce the importance of spatial parameters of the response and performance-related feedback. Configurations varied in spatial (target-penalty region overlap) and gain parameters (negative penalty values) and could have similar or different MEG. Choices were made without prior aiming experience (Experiment 1), after aiming experience provided information of movement variability (Experiment 2), or after aiming experience provided information of movement variability and outcome feedback (Experiment 3). Overall, configurations with advantageous spatial or gain parameters were chosen equally (Both-Similar condition) in all experiments. However, average behaviour at the group level was not reflective of the behaviour of most individual participants with three subgroups emerging: those with a value preference, distance preference, or no preference. In Experiments 1 and 2, these individual differences cannot be explained by MEG differences between configurations or participants' movement variability, but these variables predicted choice behaviour in Experiment 3. Further in the Both-Different condition, participants only selected the larger MEG configuration at a level above chance when both variability and outcome information were given prior to the key press task (Experiment 3). In sum, the data indicate that prioritisation of spatial information did not emerge at the group level when performing key presses and more optimal behaviour emerged when information regarding movement variability and outcome feedback were given.

Feasibility of a Mobile Health App for Routine Outcome Monitoring and Feedback in Mutual Support Groups Coordinated by SMART Recovery Australia: Protocol for a Pilot Study

BackgroundDespite the importance and popularity of mutual support groups, there have been no systematic attempts to implement and evaluate routine outcome monitoring (ROM) in these settings. Unlike other mutual support groups for addiction, trained facilitators lead all Self-Management and Recovery Training (SMART Recovery) groups, thereby providing an opportunity to implement ROM as a routine component of SMART Recovery groups.ObjectiveThis study protocol aims to describe a stage 1 pilot study designed to explore the feasibility and acceptability of a novel, purpose-built mobile health (mHealth) ROM and feedback app (Smart Track) in SMART Recovery groups coordinated by SMART Recovery Australia (SRAU) The secondary objectives are to describe Smart Track usage patterns, explore psychometric properties of the ROM items (ie, internal reliability and convergent and divergent validity), and provide preliminary evidence for participant reported outcomes (such as alcohol and other drug use, self-reported recovery, and mental health).MethodsParticipants (n=100) from the SMART Recovery groups across New South Wales, Australia, will be recruited to a nonrandomized, prospective, single-arm trial of the Smart Track app. There are 4 modes of data collection: (1) ROM data collected from group participants via the Smart Track app, (2) data analytics summarizing user interactions with Smart Track, (3) quantitative interview and survey data of group participants (baseline, 2-week follow-up, and 2-month follow-up), and (4) qualitative interviews with group participants (n=20) and facilitators (n=10). Feasibility and acceptability (primary objectives) will be analyzed using descriptive statistics, a cost analysis, and a qualitative evaluation.ResultsAt the time of submission, 13 sites (25 groups per week) had agreed to be involved. Funding was awarded on August 14, 2017, and ethics approval was granted on April 26, 2018 (HREC/18/WGONG/34; 2018/099). Enrollment is due to commence in July 2019. Data collection is due to be finalized in October 2019.ConclusionsTo the best of our knowledge, this study is the first to use ROM and tailored feedback within a mutual support group setting for addictive behaviors. Our study design will provide an opportunity to identify the acceptability of a novel mHealth ROM and feedback app within this setting and provide detailed information on what factors promote or hinder ROM usage within this context. This project aims to offer a new tool, should Smart Track prove feasible and acceptable, that service providers, policy makers, and researchers could use in the future to understand the impact of SMART Recovery groups.Trial RegistrationAustralian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12619000686101; https://anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377336.International Registered Report Identifier (IRRID)PRR1-10.2196/15113

How do different kinds of incidental emotions influence risk decision making?

Incidental emotions, which are irrelevant to the ongoing decision, play a significant role in decision-making processes. In this study, we investigated the influence of specific incidental emotions on behavioral, psychological, and electrophysiological responses during the process of decision making. Participants finished a forced-choice gambling task, during which incidental emotions (anger/fear/happiness) were elicited by recalling emotional experiences. Behavioral and event-related potential (ERP) data were recorded in the experiment. Behavioral results showed that risk preference was weaker in the fearful condition than in the angry and happy conditions, but emotional feelings to outcome feedback were not influenced by incidental emotions. The feedback-related negativity (FRN) amplitude was larger in the fearful condition than in the angry, happy, and neutral conditions for large outcomes, whereas there was no difference between the four conditions for small outcomes. In addition, the influence of outcome magnitude (small/large) on the P3 amplitude was stronger in the fearful condition than in the angry, happy, and neutral conditions. We suggest that incidental fear elicits risk-avoidant behavior (manifested in behavioral choices), stronger motivation (manifested in the FRN amplitude) and cognitive resources (manifested in the P3 amplitude) for risky outcomes of decision making unconsciously (indicated by unchanged subjective emotional experiences). The current findings have expanded our understanding of the role of specific incidental emotions in decision making.

Exploring paramedic perceptions of feedback using a phenomenological approach.

Objectives:Despite widespread advocacy of a feedback culture in healthcare, paramedics receive little feedback on their clinical performance. Provision of ‘outcome feedback’, or information concerning health-related patient outcomes following incidents that paramedics have attended, is proposed, to provide paramedics with a means of assessing and developing their diagnostic and decision-making skills. To inform the design of feedback mechanisms, this study aimed to explore the perceptions of paramedics concerning current feedback provision and to discover their attitudes towards formal provision of patient outcome feedback.Methods:Convenience sampling from a single ambulance station in the United Kingdom (UK) resulted in eight paramedics participating in semi-structured interviews. Interpretative phenomenological analysis was employed to generate descriptive and interpretative themes related to both current and potential feedback provision.Results:The perception that only exceptional incidents initiate feedback, and that often the required depth of information supplied is lacking, resulted in some participants describing an isolation of their daily practice. Barriers and limitations of the informal processes currently employed to access feedback were also highlighted. Formal provision of outcome feedback was anticipated by participants to benefit the integration and progression of the paramedic profession as a whole, in addition to facilitating the continued development and well-being of the individual clinician. Participants anticipated feedback to be delivered electronically to minimise resource demands, with delivery initiated by the individual clinician. However, a level of support or supervision may also be required to minimise the potential for harmful consequences.Conclusions:Establishing a just feedback culture within paramedic practice may reduce a perceived isolation of clinical practice, enabling both individual development and progression of the profession. Carefully designed formal outcome feedback mechanisms should be initiated and subsequently evaluated to establish resultant benefits and costs.

Effectiveness of a personalized health profile on specificity of self-management goals among people living with HIV in Canada: a protocol for a blinded pragmatic randomized controlled trial

Background: Goal setting is a crucial element in self-management of chronic diseases. Personalized outcome feedback is needed for goal setting, a requirement for behavior change. This study contributes to the understanding of the specificity of patient-formulated self-management goals by testing the effectiveness of a personalized health outcome profile. Objective: To estimate among people living with HIV, to what extent providing feedback on their health outcomes, compared to no feedback, will affect number and specificity of patient-formulated self-management goals. Methods: A personalized health outcome profile has been produced for individuals enrolled in a Canadian HIV Brain Health Now cohort study at cohort entry and at the last recorded visit. Participants will be randomized to receive or not “My Personal Brain Health Dashboard” prior to a goal setting exercise. Self-defined goals in free text will be collected through an online platform. Intervention and control groups will receive instructions on goal setting and tips to improve brain health. A total of 420 participants are needed to detect a rate ratio (number of specific words/numbers of person-goals, intervention:control) of 1.5. Text mining techniques will be used to quantify goal specificity based on word matches with a goal-setting lexicon. The expectation is that the intervention group will set more goals and have more words matching the developed lexical than the control group. The total number of words per person-goals will be calculated for each group and Poisson regression will be used to estimate the rate ratio and 95% confidence intervals and compare rate ratios between men and women using an interaction term. Conclusions: This study will contribute to growing evidence for the value of person-reported health outcomes in tailoring interventions, and will provide a thorough understanding of the quality of person-defined goals using text mining. Trial registration: Clinical Trials NCT04175795, registered on 25th November 2019.

Team Decision Making: The Dynamic Effects of Team Decision Style Composition and Performance via Decision Strategy

As teams continue to become more prevalent in modern-day organizations, researchers and organizations alike can benefit from a more nuanced understanding of teams’ decision-making process, which can ultimately impact organizational effectiveness. Although team processes are conceptualized as dynamic phenomena, they have largely been treated as static in research. In this study, we draw on the input-mediator-output-input and episodic team performance frameworks to advance a theoretical model of the dynamic, reciprocal effects of team rational decision strategy and team performance as well as the role of team composition of individual rational decision style. We sampled 320 participants in 85 teams competing in a 10-week business strategy simulation where teams made weekly strategic decisions that contributed to team performance. Teams composed of individuals with rational decision styles were more likely to adopt rational decision strategies, which led to better team performance. Additionally, results revealed a positive reciprocal effect between rational decision strategy and team performance such that teams with positive prior performance were more likely to engage in subsequent rational decision strategy. As hypothesized, team composition of members’ rational decision style was the primary determinant of team rational strategy during initial stages of team development, but the valence of outcome feedback (i.e., prior performance) took over as the stronger predictor of team rational strategy during later stages of team development. We contribute to the team and decision-making literatures by examining the dynamic process of team decision making and team performance.

Feedback in a Digital Badge Learning Experience: Considering the Instructor’s Perspective

There is growing interest in how various technical tools can be used to leverage the instructional process for both teaching and learning. Digital badges are a visual representation of learning and skills. Digital badges have been used as a way to reduce gaps in knowledge (Bowen and Thomas Change, 46(1), 21-25, 2014; Guskey Journal of Advanced Academics, 19(1), 8–31, 2007), increase engagement (Abramovich et al. Educational Technology Research and Development, 61, 217–232, 2013; Glover and Latif 2013), and develop mastery in key concepts (Academic Medicine, 88(10), 1418–1423, 2013). One challenge for educators in using digital badge systems is in how to provide detailed and specific assessment and feedback to students. A potential solution is to pair digital badges with Mastery Learning strategies. In order to examine the types and ways in which instructors provide feedback, individual instructor case studies were developed. Each case was investigated and analyzed holistically as a single entity representing the distinct evaluation style of that instructor. The coding schema used a deductive process based on feedback and Mastery Learning research including the functions of feedback (e.g. task, cognitive, and functional validity information) (Balzer et al. Psychological Bulletin, 106(3), 410, 1989; Butler and Winne Review of Educational Research, 65(3), 245–281, 1995), principles of feedback (e.g. principles supporting self-regulated learning) (Nicol & Macfarlane-Dick Studies in Higher Education, 31(2), 199–218, 2006), framework of effective feedback (e.g. content, social and interpersonal negotiation, organization and management) (Yang and Carless Teaching in Higher Education, 18(3), 285–297, 2013), and essential elements of Mastery Learning (e.g. feedback, correctives, enrichment) (Guskey Journal of Advanced Academics, 19(1), 8–31, 2007). Instructors’ feedback fell within six categories: 1) Outcome Feedback; 2) Clarification; 3) Decreasing Gaps in Knowledge; 4) Motivation and Interaction; 5) Opportunities to Further Knowledge; 6) Promotes Overall Learning and Cognitive Development. Based on the results, specific categories of feedback are discussed.

Decision-aids for non-expert decision makers

PurposeThis study extends prior accounting research on decision aids (DAs) relating to face validity. Specifically, this study aims to examine the effects of face validity through the presence of two levels of bias in DA output. The presence of bias in a DA will not affect how statistically informative an aid is but will decrease the face validity. The findings suggest that non-expert DA users recognize the bias in the DA’s suggestions as evidenced by users’ low agreement with the aid; however, they do not adjust for the bias in their performance, suggesting that non-expert users do not learn from the DA.Design/methodology/approachThis repeated-measures experimental design allows us to examine performance effects over time in response to different levels of bias in the DA output. The participants in the study are provided with outcome feedback to examine learning effects.FindingsThe findings suggest that non-expert DA users recognize the bias in the DA’s suggestions as evidenced by users’ low agreement with the aid; however, they do not adjust for the bias in their performance, suggesting that non-expert users do not learn from the DA. Although users of an unbiased DA strongly agree with the DA’s output, individual performance deteriorates over time. Initially, the users of an unbiased DA perform better than those who use a biased DA; however, over time, the performance of users of an unbiased aid deteriorates and the performance of users of the biased aid does not improve.Practical implicationsCompanies developing DAs may need to consider the effects of using a DA under circumstances different from those under which the aid was developed and that may lead to the biased DA output. This study has implications for firms that design, develop and use DAs.Originality/valueThis study considers a yet unexamined face validity issue – observable bias in DA output. This study examines deterministic DAs designed to assist the decision-maker through their ability to combine multiple cues in a systematic and consistent manner. This study has implications for firms that design, develop and use DAs. Firms need to consider the effects of using a DA under circumstances different from those under which the aid is developed, thereby, potentially leading to biased DA output. Each additional variable added to the DA will be associated with an incremental cost in a DA’s development, use and modification. The results of this study provide insights contributing to the information available for cost–benefit analyses conducted when developing a DA or when considering the modification of existing aid. Failure to change a DA because of face validity issues alone may result in a decline in user performance. Thus, the cost of modifying a DA must be weighed against the benefits resulting from improved performance. This study contributes insights into how users’ responses to DA bias could affect the assessments of the benefits of including an omitted variable in a DA.