Instrument For Evaluation Of Quality Research Articles

Online Course Quality Evaluation Instruments: A Scoping Review

How do we make judgements about the quality of online courses? Checklists and rubrics are commonplace in Higher Education for establishing and measuring design features of online courses. They are created and used by institutions, academics, and educational designers to standardise measures for quality online course design. Despite an intensifying spotlight on quality learning and teaching in Higher Education, no large-scale review of course quality instruments has occurred. This scoping review aimed to ascertain the conceptions of quality being promoted by course quality evaluation instruments and the capability-building resources that underpin these instruments.
 
 Seventy-five instruments used to measure quality in online course design in Higher Education were identified via a systematic search. These instruments were charted and coded. This paper reports on findings that summarise the key attributes of the course quality evaluation instruments, conceptualises a shared definition of course quality, and proposes specific core criteria for measuring course quality under the domains of learning design, assessment and evaluation, usability and accessibility, social interaction, and technology. This scoping review found a concerning underrepresentation of capability-building resources associated with course quality instruments and recommends the capability-building potential of these tools shifting them from compliance checkers to enable skills development.

Defining and Operationalizing Key Themes of High-quality Psychosocial Support Using the Concept Mapping Method:

Introduction:International psychosocial support guidelines reflect consensus on support principles and interventions. However, no consensus exists on what recipients consider important elements of service delivery. Within two contexts – after a potentially traumatic event (PTE) and people with Spinal Muscular Atrophy (SMA)–the aims were to contribute to (1) understanding which psychosocial support aspects are considered important by recipients and relevant stakeholders; (2) developing instruments to test and integrate those aspects in practice, in order to evaluate the quality of psychosocial support from the recipient’s perspective.Method:Concept mapping was used to achieve consensus on key themes of psychosocial support. These were operationalized in surveys and pilot-tested, conforming to the Consumer Quality Index. This determines the importance and needs for improvement.Results:Concept mapping resulted in eight key themes within the PTE context and six in the SMA context. PTE survey (N= 132) results showed key themes “an approach that starts from the needs and capacities of the affected one” and “monitoring individuals affected and initiating follow-up where needed” were most important. Key theme “providing information on common emotional reactions” received the highest score of perceived need for improvement. SMA survey (N= 57) results showed key themes “an approach that incorporates all aspects of a human being” and “a respectful approach and awareness of personal boundaries” as most important. The perceived need for improvement of the key theme “availability and accessibility of quality information” was ranked the highest.Conclusion:The similarities between both contexts support the notion that there are universal aspects of psychosocial support. Simultaneously, the context-specific idiosyncrasies found underscore the necessity to adapt to context. The surveys have the potential to contribute to a growing toolbox of quality evaluation instruments.

Social media in health communication: A literature review of information quality.

Social media is used in health communication by individuals, health professionals, disease centres and other health regulatory bodies. However, varying degrees of information quality are churned out daily on social media. This review is concerned with the quality of Social Media Health Information (SMHI). The review sought to understand how SMHI quality issues have been framed and addressed in the literature. Health topics, users and social media platforms that have raised health information quality concerns are reviewed. The review also looked at the suitability of existing criteria and instruments used in evaluating SMHI and identified gaps for future research. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the forward chaining strategy were used in the document search. Data were sourced according to inclusion criteria from five academic databases, namely Scopus, Web of Science, Cochrane Library, PubMed and MEDLINE. A total of 93 articles published between 2000 and 2019 were used in the review. The review revealed a worrying trend of health content and communication on social media, especially of cancer, dental care and diabetes information on YouTube. The review further discovered that the Journal of the American Medical Association, the DISCERN and the Health on the Net Foundation, which were designed before the advent of social media, continue to be used as quality evaluation instruments for SMHI, even though technical and user characteristics of social media differ from traditional portals such as websites. The study synthesises varied opinions on SMHI quality in the literature and recommends that future research proposes quality evaluation criteria and instruments specifically for SMHI.

Анкетирование как инструмент оценивания качества педагогической деятельности в университетской системе повышения квалификации по английскому языку

Анкетирование как инструмент оценивания качества педагогической деятельности в университетской системе повышения квалификации по английскому языку

The quality of non-alcoholic fatty liver disease information resources for patients on the internet in Korea.

Background/AimsMany patients uses the internet to obtain information about their diseases. However, there is increasing concern regarding the quality of internet information. Thus, we aimed to systematically evaluate the quality of websites containing educational information about non-alcoholic fatty liver disease (NAFLD) in Korea.MethodsNaver, Daum, and Google search engines were searched using the term “non-alcoholic fatty liver disease” in Korean. Two reviewers independently evaluated website quality using the quality evaluation instrument (QEI), which awarded websites scores for specific information on various aspects of NAFLD, as well as a five-point Likert scale (1–5), the DISCERN instrument, and a global quality scale (GQS).ResultsForty-seven websites met the inclusion criteria. We found that the quality of the internet information about NAFLD is generally poor. The mean QEI score with standard deviation was 10.31 ± 5.09 (range, 4 to 22), with only 17% of websites scoring higher than 10 points. The median GQS of the websites was 2.0, with no website achieving a score of 4 or 5. The QEI score was highly associated with the GQS score (r = 0.74, p < 0.01). For each DISCERN question from question 1 to question 15, the mean score was less than 3.ConclusionsOverall, the internet health information for patients regarding NAFLD is poor and in need of much improvement. There is a need for institutional support, qualitative regulation of internet information, and development of an accreditation system to provide patients with internet health information of appropriate quality.

The Evaluation Inclusive Education Program Based Pancasila Values in Giwangan Elementary School Yogyakarta

This research is educational research and development. The aim is developing an&#x0D; evaluation instrument that implementation inclusive education based on the values of&#x0D; Pancasila in elementary school. The research target is to get the evaluation instrument&#x0D; for inclusion school so that the known effectiveness of inclusive education based on the&#x0D; values of Pancasila in elementary school. Research methods have used research and&#x0D; development. Data collection techniques using an instrument. The object of research&#x0D; is the Giwangan elementary school. The results showed the steps used in developing&#x0D; the evaluation instrument of inclusive education program is analyzing needs, designing&#x0D; development, producing/developing evaluation instrument, to evaluate products, and&#x0D; do the final product testing and quality evaluation instrument of inclusive education&#x0D; program based on the values of Pancasila. Thus, this evaluation instrument can be used&#x0D; to conduct the evaluation in school which would program the inclusive educational&#x0D; based on the values of Pancasila, particularly on the first principle, second principle,&#x0D; and the fifth principle.

Forecasting effect of Caprini risk assessment model for venous thromboembolism on venous thromboembolism among surgery patients: a Meta analysis

Objective To comprehensively review the forecasting effect of Caprini risk assessment model for venous thromboembolism (VTE) on VTE among surgery patients with the method of Meta analysis. Methods We retrieved the articles on the forecasting Caprini risk assessment model for VTE on VTE in the Cochrance Library, PubMed, Web of Science, EMBASE, China Biology Medicine, China National Knowledge Infrastructure (CNKI) , WanFang data, Vip Database (VIP) published from establishing database to 31st December 2017. The diagnostic studies quality evaluation instrument was used to assess the quality of the included articles and the data were analyzed with the Meta-Disc 1.4 software and Stata software. Results A total of 7 articles were included (n=1 520) . Heterogeneity test showed that there was heterogenicity in the included researches with 0.85 (95% CI: 0.79-0.90) for the combined sensibility by random effect model, 0.58 (95% CI: 0.55-0.61) for the specificity, 2.16 (95% CI: 1.76-2.65) for the positive likelihood ratio, 0.18 (95% CI: 0.07-0.48) for the negative likelihood ratio and 12.06 (95% CI: 4.76-30.56) . The area under the curve of summary receive opearating characteristic curve (SROC) was 0.748 with no publication bias. Conclusions The Caprini risk assessment model for VTE is a single indicator for the forecasting effect of the VTE of patients with surgery along with a high predictive sensibility and a strong overall diagnostic efficiency. It can effectively screen patients with the risk of venous thrombosis with a diagnostic value and low specificity. We should carry out many perspective studies and establish a suitable critical value aiming at scales in the future. Key words: Meta-analysis; Venous thromboembolism; Caprini risk assessment model for venous thromboembolism; Forecasting; Systematic review

Validation of the Glaucoma Quality of Life-15 Questionnaire in Serbian language.

To translate the Glaucoma Quality of Life-15 (GQL-15) to Serbian language and asses its validity and reliability in the population of Serbian patients. The study included 177 glaucoma patients. Clinical parameters (visual acuity, mean defect and square root of loss variance of visual field) and socio-demographic data were collected. Patients were stratified according to the Nelson's glaucoma staging system as mild, moderate and advanced glaucoma. All patients filled out the GQL-15 and National Eye Institute Visual Function Questionnaire (NEI-VFQ 25). The GQL-15 was translated following the internationally-accepted methodology, and its psychometric properties were assessed by using classical test theory and Rasch analysis. The mean total score for the GQL-15 was 20.68±7.31. The Cronbach's alpha coefficient for the whole scale was 0.89 (central and near vision, α=0.24; peripheral vision, α=0.85; glare and dark adaptation, α=0.83). Factor analysis established 4 factors (70.3% of variance): two corresponding to the original factors and two new factors specific for the Serbian population. The GQL-15 score correlated positively with almost all clinical parameters and NEI-VFQ 25 proving good criterion validity. Correlation of the GQL-15 total score on test-retest confirmed appropriate scale reproducibility (ρ=0.96, P<0.001). The GQL-15 discriminated well advanced from mild and moderate glaucoma. In Rasch analysis we obtained adequate item (0.95) reliability index. Almost all items had infit and outfit mean squares in the accepted range. Serbian version of the GQL-15 demonstrates adequate reliability and validity. This version of the GQL-15 is a valid instrument for evaluation of quality of life among Serbian speaking patients with glaucoma and can be applied in daily clinical work.

The Social Media Index as an Indicator of Quality for Emergency Medicine Blogs: A METRIQ Study

Online educational resources such as blogs are increasingly used for education by emergency medicine clinicians. The Social Media Index was developed to quantify their relative impact. The Medical Education Translational Resources: Indicators of Quality (METRIQ) study was conducted in part to determine the association between the Social Media Index score and quality as measured by gestalt and previously derived quality instruments. Ten blogs were randomly selected from a list of emergency medicine and critical care Web sites. The 2 most recent clinically oriented blog posts published on these blogs were evaluated with gestalt, the Academic Life in Emergency Medicine Approved Instructional Resources (ALiEM AIR) score, and the METRIQ-8 score. Volunteer raters (including medical students, emergency medicine residents, and emergency medicine attending physicians) were identified with a multimodal recruitment methodology. The Social Media Index was calculated in February 2016, November 2016, April 2017, and December 2017. Pearson's correlations were calculated between the Social Media Index and the average rater gestalt, ALiEM AIR score, and METRIQ-8 score. A total of 309 of 330 raters completed all ratings (93.6%). The Social Media Index correlated moderately to strongly with the mean rater gestalt ratings (range 0.69 to 0.76) and moderately with the mean rater ALiEM AIR score (range 0.55 to 0.61) and METRIQ-8 score (range 0.53 to 0.57) during the month of the blog post's selection and for 2 years after. The Social Media Index's correlation with multiple quality evaluation instruments over time supports the hypothesis that it is associated with overall Web site quality. It can play a role in guiding individuals to high-quality resources that can be reviewed with critical appraisal techniques.

P022: The revised METRIQ Score: an international, social-media based usability analysis of a quality evaluation instrument for medical education blogs

Introduction: Online medical education resources are widely used in emergency medicine (EM), but strategies to assess quality remain elusive. We previously derived the Medical Education Translational Resources: Impact and Quality (METRIQ) 8 instrument to evaluate quality in medical education blog posts. Methods: As part of a subsequent validation study (The METRIQ Blog Study), a mixed-methods usability analysis was performed to obtain user feedback on the quality assessment instrument in order to improve its clarity and reliability. Participants in the METRIQ Study were first asked to rate five blog posts using the METRIQ-8 Score. They then evaluated the METRIQ-8 instruments ease of use and likelihood of being recommended to others using a 7-point Likert scale and free text comments. Participants were also asked to flag and comment on items within the score that they felt were unclear. Global usability ratings were summarized using median scores or percent rated unclear. We used ANOVA to test associations between ease of use and demographic factors. A thematic analysis was performed on the comments. Results: 309 EM medical students, residents, and attendings completed the survey. Global ratings were generally very favorable (median 2 [IQR 2-3], with 7 being the lowest score) for ease of use and likelihood of recommendation, and did not vary by participants country of origin, frequency of blog use, or learner level. Participants stated that the score was structured, systematic, and straightforward. They found it useful for junior learners and for guiding blog creation. Four questions in the score (questions 2, 4, 5, and 7) were identified by 10% of subjects to be unclear. Thematic analysis of comments identified suggested four main themes for improving the score: adding clearer definitions with marking rubrics; shortening the 7-point scale; adding items evaluating blog post presentation and utility; and, rephrasing the wording of certain questions for clarity. Conclusion: A mixed methods usability analysis of the METRIQ-8 instrument for assessing blog quality was globally well received by EM medical students, residents, and attendings. Qualitative analyses revealed multiple areas to improve the instruments clarity and usability. The METRIQ score is a promising instrument for evaluating the quality of blogs; further development and testing is needed to improve its utility.

Developing quality evaluation instrument for Islamic Senior High School

This research aims to develop concept of the quality of education in Islamic Senior High School and know the validity and reliability of the factors that affect the development of quality indicators in Islamic Senior High School. Development of the procedures in this research included a needs analysis, product development, product test, and implementation/ dissemination. A sample of this research was Islamic Senior High School 1 Surakarta and Islamic Senior High School 1 Purwodadi, which were determined by a purposive random sampling. Data analysis techniques used was Confirmatory Factor Analysis (CFA), Lisrel version 8.54, and descriptive qualitative analysis. The results of the study showed that the development of quality indicator for Islamic Senior High School includes six criteria namely, the role of school principals, management of Islamic Senior High School, educators and Staffs, Students, facilities and infrastructures and the role of the community. It has met the indicators fit models, the Chi-Square value is 152.07, p &gt; 0.05, and RMSEA is 0.068. The advantages of this Islamic Senior High School Quality Evaluation Instrument are namely comprehensive holistic approach, easy, not depending on the specific lessons conducted by educators. It can be used to map the various aspects of the education in Islamic school that need to be repaired; it is open to further developed) Islamic school quality evaluation instrument can be used as a school self-evaluation; it can be used effectively by schools without interrupting the learning process; it supports school accreditation and quality guarantee, and it is also independent.

Derivation of Two Critical Appraisal Scores for Trainees to Evaluate Online Educational Resources: A METRIQ Study

IntroductionOnline education resources (OERs), like blogs and podcasts, increasingly augment or replace traditional medical education resources such as textbooks and lectures. Trainees’ ability to evaluate these resources is poor, and few quality assessment aids have been developed to assist them. This study aimed to derive a quality evaluation instrument for this purpose.MethodsWe used a three-phase methodology. In Phase 1, a previously derived list of 151 OER quality indicators was reduced to 13 items using data from published consensus-building studies (of medical educators, expert podcasters, and expert bloggers) and subsequent evaluation by our team. In Phase 2, these 13 items were converted to seven-point Likert scales used by trainee raters (n=40) to evaluate 39 OERs. The reliability and usability of these 13 rating items was determined using responses from trainee raters, and top items were used to create two OER quality evaluation instruments. In Phase 3, these instruments were compared to an external certification process (the ALiEM AIR certification) and the gestalt evaluation of the same 39 blog posts by 20 faculty educators.ResultsTwo quality-evaluation instruments were derived with fair inter-rater reliability: the METRIQ-8 Score (Inter class correlation coefficient [ICC]=0.30, p<0.001) and the METRIQ-5 Score (ICC=0.22, p<0.001). Both scores, when calculated using the derivation data, correlated with educator gestalt (Pearson’s r=0.35, p=0.03 and r=0.41, p<0.01, respectively) and were related to increased odds of receiving an ALiEM AIR certification (odds ratio=1.28, p=0.03; OR=1.5, p=0.004, respectively).ConclusionTwo novel scoring instruments with adequate psychometric properties were derived to assist trainees in evaluating OER quality and correlated favourably with gestalt ratings of online educational resources by faculty educators. Further testing is needed to ensure these instruments are accurate when applied by trainees.

Qualität der physiotherapeutischen Dokumentation in der Patientenkrankengeschichte

Background: This article investigated a Dutch scientific study on the evaluation of the quality of physiotherapy patient documentation and its implementation in daily practise of physiothera- pists in Switzerland. Objective: To find a standardised procedure within the scope of a quality-circle in order to as- sess and identify flaws and improve the actual state of physiotherapeutic patient documenta- tion, then test the procedure and advance an opinion about it. Method: The quality evaluation instrument in- cluding the corresponding article's summary and quintessence were translated into German. A non-member's back-translation ensured the indi- cators' analogous rendition in German. After- wards all members evaluated several patient charts by means of this instrument. The applica- tion served as a basis for the discussion about the quality evaluation instrument and all the item- ized indicators. Results: All indicators can be transferred to Swiss conditions. The instrument is well suited for qual- ity survey of the physiotherapy patient documen- tation within the scope of a quality-circle. It offers the opportunity to implement specific desirable or necessary corrections in relevant clinical de- tails. As an individual aim 2 or 3 indicators can offhand be identified. Conclusion: The quality evaluation instrument should also be available to other physiotherapists, moderators and their quality-circles. With the authors' consent it was decided to offer the origi- nal article as well as the manual and score-form as download on the Swiss Association of Physio- therapy website (www.physioswiss.ch).

Validity and Reliability of the Korean Version of the Observable Indicators of Nursing Home Care Quality Evaluation Instrument

The purposes of this study were; 1) to test the validity and reliability of the Korean version of Rantz's Observable Indicators of Nursing Home Care Quality Instrument (ONHQ) and 2) to evaluate the quality of Korean nursing homes. The study employed a three-phase methodological research design. 1) The original instrument of Rantz's ONHQ was translated into Korean and modified by Korean nursing home experts. 2) A pilot study using the modified instrument was done in 20 nursing homes to examine inter-rater reliability. 3) The validity and reliability were tested in 98 nursing homes. Seven factors were extracted through factor analysis: 'communication', 'care delivery', 'grooming', 'odor', 'environment-basics', 'environment-access', and 'environment-homelike'. These factors explained 86.07% of the total variance. Cronbach's alpha coefficient of the 30 items was .97 indicating a high internal consistency of the instrument. Inter-rater reliability according to Kappa was .82. The average score of nursing home quality was 112.07 indicating an average range of quality level. The Korean version of the ONHQ was identified as a tool with a high degree of validity and reliability. This tool can be effectively used to assess the quality of nursing homes by professions as well as family members.

A Systematic Review of Patient Inflammatory Bowel Disease Information Resources on the World Wide Web

The Internet is a widely used information resource for patients with inflammatory bowel disease, but there is variation in the quality of Web sites that have patient information regarding Crohn's disease and ulcerative colitis. The purpose of the current study is to systematically evaluate the quality of these Web sites. The top 50 Web sites appearing in Google using the terms "Crohn's disease" or "ulcerative colitis" were included in the study. Web sites were evaluated using a (a) Quality Evaluation Instrument (QEI) that awarded Web sites points (0-107) for specific information on various aspects of inflammatory bowel disease, (b) a five-point Global Quality Score (GQS), (c) two reading grade level scores, and (d) a six-point integrity score. Thirty-four Web sites met the inclusion criteria, 16 Web sites were excluded because they were portals or non-IBD oriented. The median QEI score was 57 with five Web sites scoring higher than 75 points. The median Global Quality Score was 2.0 with five Web sites achieving scores of 4 or 5. The average reading grade level score was 11.2. The median integrity score was 3.0. There is marked variation in the quality of the Web sites containing information on Crohn's disease and ulcerative colitis. Many Web sites suffered from poor quality but there were five high-scoring Web sites.

Quality of MPEG2 signal on a simulated digital terrestrial television

The monitoring of the quality of service in digital television networks needs instruments to assess automatically the perceived quality of audio and video components of an MPEG2 signal. In order to improve the knowledge of the impact of the impairments on the final service, and to validate the algorithms that will be implemented in the quality evaluation instruments, a general simulation chain has been designed and developed. This paper presents the simulations performed as well as the analysis and the synthesis of the obtained results.