Questionnaire Completion Time Research Articles

Development of a brief, generic, modular resource-use measure (ModRUM): piloting with patients

BackgroundBespoke self-report resource-use measures (RUMs) are commonly developed or adapted for each new randomised controlled trial. Consequently, RUMs lack standardisation and validation is rarely conducted. A new generic RUM, ModRUM, has been developed using a rigorous process, including consultation with health economists and patients. ModRUM includes a concise core healthcare module, designed to be included in all trials, and depth-adding questions, which can replace or be added to core questions as needed. Modules covering other sectors are under development. The aim of this study was to test the acceptability, feasibility, and criterion and construct validity of the healthcare module of ModRUM.MethodsPatients who had a recent appointment at their GP practice were invited to complete ModRUM (core module or core module with depth questions), a characteristics form and the EQ-5D-5L. Acceptability was assessed via response rates and questionnaire completion time. Feasibility was assessed by reviewing issues observed in participants’ responses and question completion rates. Construct validity was tested via hypothesis testing and known-group analyses, using Wilcoxon rank-sum and Kruskal–Wallis tests, and a generalised linear model. Criterion validity was tested by comparing ModRUM results with primary care medical records. Sensitivity, specificity, and agreement using Lin’s concordance correlation coefficient (pc) were estimated.ResultsOne hundred patients participated from five GP practices in the South-West of England. Acceptability was higher for the core module (20% versus 10% response rate). Question completion rates were high across both versions (> 90%). Some support was observed for construct validity, with results suggesting that healthcare costs differ dependent on the number of long-term conditions (p < 0.05) and are negatively associated with health-related quality of life (p < 0.01). Sensitivity was high for all questions (> 0.83), while specificity varied (0.33–0.88). There was a good level of agreement for GP contacts and costs, and prescribed medication costs (pc > 0.6).ConclusionThis study provided preliminary evidence of the acceptability, feasibility, and criterion and construct validity of ModRUM. Further testing is required within trials and with groups that were less well represented in this study.

Rating versus ranking in a Delphi survey: a randomized controlled trial

BackgroundThe Delphi technique has steeply grown in popularity in health research as a structured approach to group communication process. Rating and ranking are two different procedures commonly used to quantify participants’ opinions in Delphi surveys. We explored the influence of using a rating or ranking approach on item prioritization (main outcome), questionnaire completion time, and evaluation of task difficulty in a Delphi survey aimed at identifying priorities for the organization of primary cardiovascular care.MethodsA randomized controlled parallel group trial was embedded in a three-round online Delphi survey. After an “open” first round, primary care patients, trained patient partners, and primary care clinicians from seven primary care practices were allocated 1:1 to a rating or ranking assessment group for the remainder of the study by stratified permuted block randomization, with strata based on participants’ gender and status. Agreement on item prioritization between the experimental groups was measured by calculating Krippendorff’s alpha reliability coefficient on the aggregate rank order of items in each group after the final round. Self-reported ease or difficulty with the assessment task was measured with the Single Ease Question.ResultsThirty-six panelists (13 clinic patients, 7 patient partners, 16 clinicians; 60% females) were randomized to the rating (n = 18) or ranking (n = 18) group, with 30 (83%) completing all rounds. Both groups identified the same highest priorities from a set of 41 items, but significant discrepancies were found as early as the seventh top item. There was moderately strong agreement between the priority ordering of top items common to both groups (Krippendorff’s alpha = 0.811, 95% CI = 0.669–0.920). A 9-min mean difference to complete the third-round questionnaire in favor of the rating group failed to achieve statistical significance (p = 0.053). Ranking was perceived as more difficult (p < 0.001).ConclusionsA rating or ranking procedure led to modestly similar item prioritization in a Delphi survey, but ranking was more difficult. This study should be replicated with a larger number of participants and with variations in the ranking and rating procedures.Trial registrationNot applicable.

Questionnaire experience of the pictorial usability inventory (PUI) – a comparison of pictorial and hybrid usability scales

In recent years, alternative types of usability questionnaires using graphical elements (pictorial scales) or a combination of graphical and verbal elements (hybrid scales) have been introduced. Previous research indicates that these questionnaires have advantages, such as increased respondent motivation, and drawbacks, such as extended questionnaire completion time. This study aimed to systematically investigate the psychometric properties and the respondents’ experience of two versions of a recently developed questionnaire, the Pictorial Usability Inventory (PUI), consisting of a hybrid and pictorial version. Given that questionnaire length is a crucial factor for the usefulness of a scale, the study tested long and short versions (8 items vs 3 items) of both questionnaire types. The study involved an online usability test with 777 participants, who were asked to complete one of the four PUI versions and an established verbal usability scale after solving three tasks on a webpage. The results demonstrated high sensitivity, high convergent validity, and good internal consistency for all four PUI versions. While the long pictorial scale achieved the best psychometric properties overall, participants preferred the hybrid scales, particularly the short version. The study's findings are in line with previous research on pictorial and hybrid instruments and suggest that hybrid instruments, particularly short ones, may be superior to purely pictorial instruments in terms of respondent-centred aspects conceptualised in the term ‘questionnaire experience’.

Development and validation of the thyroid cancer self-perceived discrimination scale to identify patients at high risk for psychological problems.

To develop a Thyroid Cancer Self-Perceived Discrimination Scale (TCSPDS) to identify patients at high risk for psychological problems and to test its reliability, validity and acceptability. Using classical test theory, a total of 176 thyroid cancer patients from November 2021 to October 2022 were recruited to develop the TCSPDS. Item analysis was used to improve the preliminary TCSPDS. Exploratory factor analysis (EFA), confirmatory factor analysis (CFA) and structural equation model (SEM) were used to test the construct validity of the final TCSPDS. Pearson correlation coefficient was used to analyze the validity coefficient between TCSPDS and EORTC QLQ-C30 to test the criterion-related validity (CRV) of the final TCSPDS. The internal consistency coefficient (Cronbach's alpha coefficient), split half reliability (Spearman-Brown coefficient) and test-retest reliability were used to verify the reliability of the final TCSPDS. The questionnaire completion time and effective response rate were used to validate the acceptability of the final TCSPDS. The TCSPDS consisted of 20 items and was divided into 3 subscales: 8 items for stigma, 6 items for self-deprecation, and 6 items for social avoidance. The TCSPDS had good validity (χ2/df=1.971, RMSEA=0.074, GFI=0.921, CFI= 0.930, IFI=0.932, TLI=0.901, Validity coefficient=0.767), reliability (Cronbach's alpha=0.867, Spearman-Brown coefficient=0.828, test-retest reliability coefficient=0.981) and acceptability [average completion time (15.01 ± 1.348 minutes) and an effective response rate of 95.14%]. Patients with higher TCSPDS scores reported a lower quality of life (P<0.05). The TCSPDS could be used for early identification and assessment of the level of self-perceived discrimination in patients with thyroid cancer, which may provide a scientific basis for health education, social support and psychosocial oncology services in the future, especially in Southwest China.

Investigating the impact of demographic and academic variables on assessing students' perceived information literacy self-efficacy

This paper explores the demographic and academic variables influencing students' perceived information literacy self-efficacy using Kurbanoglu et al.'s Information Literacy Self-Efficacy Scale (ILSES) and its two subscales. An online survey was created to gather data from students which contained Bangla version of the 28-item ILSES together with demographic and academic questions. A printed version of the survey was also distributed to students locally. A total of 408 students participated in the study. The findings revealed that students had moderate self-efficacy, with higher confidence in basic and intermediate-level information literacy (IL) tasks compared to advanced skills. There were significant differences in IL self-efficacy based on students' demographic and academic variables. The results suggest that age and computer proficiency were significantly associated with students' self-efficacy beliefs in information literacy. Overall, the 28-item ILSES and its two smaller subscales yielded similar results, indicating that shorter versions of the ILSES can be utilized to reduce questionnaire completion time without compromising the validity of the results. This study highlights the importance of demographic and academic variables in shaping students' IL self-efficacy beliefs and emphasizes the need for providing adequate support and training to improve information skills among students.

The Role of Web-Based Adaptive Choice-Based Conjoint Analysis Technology in Eliciting Patients’ Preferences for Osteoarthritis Treatment

To assess the feasibility of using adaptive choice-based conjoint (ACBC) analysis to elicit patients' preferences for pharmacological treatment of osteoarthritis (OA), patients' satisfaction with completing the ACBC questionnaire, and factors associated with questionnaire completion time. Adult patients aged 18 years and older with a medical diagnosis of OA, experiencing joint pain in the past 12 months, and living in the Northeast of England participated in the study. The participants completed a web-based ACBC questionnaire about their preferences regarding pharmaceutical treatment for OA using a touchscreen laptop independently, and accordingly, the questionnaire completion time was measured. Moreover, the participants completed a pen-and-paper feedback form about their experience in completing the ACBC questionnaire. Twenty participants aged 40 years and older, 65% females, 75% had knee OA, and suffering from OA for more than 5 years participated in the study. About 60% of participants reported completing a computerized questionnaire in the past. About 85% of participants believed that the ACBC task helped them in making decisions regarding their OA medications, and 95% agreed or strongly agreed that they would be happy to complete a similar ACBC questionnaire in the future. The average questionnaire completion time was 16 min (range 10-24 min). The main factors associated with longer questionnaire completion time were older age, never using a computer in the past, and no previous experience in completing a questionnaire. The ACBC analysis is a feasible and efficient method to elicit patients' preferences for pharmacological treatment of OA, which could be used in clinical settings to facilitate shared decision-making and patient-centered care. The ACBC questionnaire completion consumes a significantly longer time for elderly participants, who never used a computer, and never completed any questionnaire previously. Therefore, the contribution of patients and public involvement (PPI) group in the development of the ACBC questionnaire could facilitate participants' understanding and satisfaction with the task. Future research including patients with different chronic conditions may provide more useful information about the efficiency of ACBC analysis in eliciting patients' preferences for osteoarthritis treatment.

Exploring Validated Self-Reported Instruments to Assess Adherence to Medications Used: A Review Comparing Existing Instruments.

ObjectiveTo provide an overview of instruments that measure adherence to medications to facilitate selection of an instrument and to provide a summary of some published adherence assessment tools tailored to be used by the healthcare team, especially the pharmacists.Data SourcesStudies were identified via PubMed, Medline, Embase, and Google Scholar using the search terms medication adherence, compliance, persistence, combined with the terms questionnaire, scale, survey, self-report, and instrument.Study SelectionArticles written in English, describing questionnaire instruments that were psychometrically evaluated (reporting a good coefficient of internal consistency, reliability assessed through Cronbach’s alpha, and had been validated against a subjective or objective measure), and containing 30 or fewer items were included.Data SynthesisTwelve instruments were identified and included in this review. Instruments were reviewed by evaluating specific characteristics (number of items, sample size, reliability, sensitivity, specificity, questionnaire completion time, criterion validity, and component analysis).ConclusionVarious instruments are used to assess patient adherence to medications; however, no single one performs well on all criteria. Every instrument has different advantages and unique properties. A particular instrument can be chosen after considering certain factors, such as the specific requirements, population, the needed time to complete the questionnaire, the sensitivity, and the specificity of the questionnaire. Moreover, there is a recognized need to provide primary care medication adherence services customized to patient’s needs. A link for teamwork between healthcare providers such as pharmacists and patients is needed. This link can be a validated instrument to assess patient’s adherence to medication.

Are Abbreviated Mental Test (AMT) and Ascertain Dementia 8 Indonesia (AD8-INA) Questionnaires More Superior than Mini-Mental State Examination (MMSE) as Dementia Screening Instrument among Elderly in Rural Areas?

Introduction: Dementia screening provides numerous benefits to its users. However, current screening methods have several limitations regarding applicability and accuracy, making it difficult to accommodate the results. Objective: To describe whether the Abbreviated Mental Test (AMT) and Ascertain Dementia 8 Indonesia (AD8-INA) questionnaire is superior to Mini-Mental State Examination (MMSE) questionnaire as a dementia screening instrument for the elderly in rural areas. Methods: A cross-sectional study was conducted in February 2020 at Banyuwangi residence. Dementia screening was conducted among elderly respondents using MMSE, AMT, and AD8-INA questionnaires. Sensitivity and specificity of AMT, AD8-INA and combined AMT+AD8-INA were compared with the MMSE questionnaire using crosstabs. Comparison of time required to complete each questionnaire was analyzed using Wilcoxon Signed Rank Test. Results: Mean age among 59 respondents was 68.44 years. The average MMSE score was 24.54. Compared with MMSE, the AMT questionnaire had a sensitivity of 47.37% and specificity of 100% (X2 = 22.36, p &lt;0.001). Meanwhile, the AD8-INA questionnaire had a sensitivity of 63.16% and specificity of 45% (X2 = 24.64, p &lt;0.001). The average questionnaire completion time of AMT, AD8-INA, and combined AMT+AD8-INA each was significantly shorter than MMSE (122.59, 121.17, and 243.76 seconds vs 319.83 seconds, p&lt;0.001, &lt;0.001, and &lt;0.001, respectively). Conclusion: This study found that the combined AMT+AD8-INA questionnaire could be used as a dementia screening instrument among the elderly in rural areas with considerable sensitivity and shorter administration time.

Using pens as an incentive for questionnaire return in an orthopaedic trial: an embedded randomised controlled retention trial

Background: We did a ‘study within a trial’ (SWAT), evaluating the effectiveness of the inclusion of a pen with a postal questionnaire, compared to no pen being included, on the retention rate in a large orthopaedic trial. Methods: The SWAT was embedded in the KReBS trial. The primary outcome was the proportion of 12-month questionnaires returned. Secondary outcomes were the proportion of questionnaires completed and time to questionnaire return. Binary data were analysed using logistic regression and time to return using Cox proportional hazards regression. Odds ratios (OR) and hazard ratios (HR) are presented, with associated 95% confidence intervals and p-values. Results: In total, 2305 participants were randomised into the SWAT. In the pen group, 1020/1145 (89.1%) of participants returned a questionnaire, compared to 982/1147 (85.6%) in the no pen group. The absolute difference in questionnaire return rate was 3.5% (95% CI: 0.8% to 6.2%; p=0.01). There were statistically significant differences in questionnaire return rate (OR 1.36; 95% CI: 1.06 to 1.74; p=0.02), questionnaire completion rate (OR 1.40; 95% CI: 1.11 to 1.78; p&lt;0.01) and time to questionnaire return (HR 1.17; 95% CI: 1.07 to 1.27; p&lt;0.01) favouring the pen group. Conclusion: This SWAT adds to the growing evidence base for whether pens are effective as an incentive for retention, and indicates their potential effectiveness. Registration: KReBS trial registered on 20 February 2019, ID ISRCTN87127065; SWAT registered on 1 April 2019, ID SWAT92.

POS0152-HPR REMOTE CLINICAL MANAGEMENT: INCORPORATING ELECTRONIC ASSESSMENT OF PATIENTS WITH RHEUMATIC DISEASES INTO STANDARD CLINICAL PATHWAYS DURING THE COVID-19 PANDEMIC: A PILOT STUDY

Background:Many patients with rheumatic disease require immunosuppressive medication putting them at high risk of COVID-19 infection. Reduced staffing in rheumatology due to redeployment to COVID-19 work, limited out patient capacity and patient vulnerability have had a major impact on our ability to review our patients to assess their condition and treatment (by face-to-face, video or telephone consultations). Novel strategies are essential to safely and effectively treat patients with rheumatic disease whilst minimising their risk of exposure to COVID-19 infection.Objectives:The objective was to develop a digital solution to help deliver safe, efficient and effective care for patients with rheumatic diseases. The aim was to produce a system that allowed us to integrate data recorded directly by patients with information held in our electronic health records to provide a virtual review of care.Methods:An online questionnaire was used to collect clinical information, including validated disease activity measures, to conduct a remote assessment in 175 patients awaiting follow-up appointments. This assessment was integrated within our electronic health records (EHR). The questionnaire contained measures of disease activity (DAS28 or BASDAI); patient reported outcomes; patient preferences regarding the urgency and type of appointment; any recent problems or changes in medication. This information was imported into a database for clinician review, together with previous clinical records and results of relevant investigations, to inform clinical decisions and to decide on the safest and most appropriate timing for follow-up. Report letters were sent to the patient and their primary care providers.Results:Of the 175 patients (149 with RA and 26 with AS), 108 patients (89/149 [60%] with RA [mean age=64; female=65%] and 19/26 [73%] with AS [mean age=45; female=54%]) submitted responses over a 6-week period based on which clinical decisions were made. The mean questionnaire completion time was 19 minutes for RA responders and 16 minutes for AS responders. Non responders (67/175 [mean age=61; female=63%]) remained on our list of patients awaiting follow-up arrangements to be made. Sixty-nine responders (64%) had stable disease therefore did not require any changes to their treatment and were offered an appointment within the next 6 months, of whom 12 (11%) requested face-to-face follow-up. Of the remaining 39 – with less stable disease – requiring more rapid follow-up assessment, 22 patients (56%) required a face-to-face consultation to consider treatment change. So far 9 of these patients have had follow-up, of whom 6 necessitated treatment escalation (Methotrexate increase n=2; anti-inflammatory increase n=2; intramuscular steroid n=1; anti-TNF escalation n=1). Thirty-nine patients (36%) provided feedback on the process of completing the questionnaire, 85% of whom used a mobile phone and the remainder used a computer or tablet. The majority (70%) found it “extremely easy” or “somewhat easy” to complete; remaining responses: “neutral” 20%, “somewhat difficult” 10%, “extremely difficult” 0%.Conclusion:We have created and tested a system of remote clinical management for patients with RA and AS. Amongst the 108 responders, just 31% required a face-to-face appointment, with treatment changes made accordingly. With a backlog of 3,800 awaiting allocation to follow-up appointments, remote clinical management will allow us to safely and efficiently prioritise patients requiring urgent follow-up for treatment optimisation. We will integrate this system into our standard care pathway beyond the COVID-19 pandemic to streamline our service, deliver effective care and provide evidence to support the use of costly biologic drugs.1 We plan to investigate the barriers for non-responders.

Questionnaire experience and the hybrid System Usability Scale: Using a novel concept to evaluate a new instrument

This article presents the concept of questionnaire experience (QX), intending to add a new element to the psychometric evaluation of questionnaires, which may eventually help increase the validity and reliability of instruments. The application of QX is demonstrated in the development of the Hybrid System Usability Scale (H-SUS), making use of items comprising pictorial and verbal elements to measure perceived usability. The H-SUS was modelled on the verbal version of the System Usability Scale (SUS). Since previous research showed advantages of pictorial scales over verbal scales (e.g., higher respondent motivation) but also disadvantages (e.g., longer completion times), we assumed that hybrid scales would combine the advantages of both scale types. The goal of this study was to compare the two instruments by assessing traditional psychometric criteria (convergent, divergent and criterion-related validity, reliability and sensitivity) and respondent-related aspects of QX (respondent workload, respondent motivation, questionnaire preference, and questionnaire completion time). An online experiment was carried out (N = 152), in which participants interacted with a smartphone prototype and subsequently completed the verbal SUS together with the H-SUS. Results indicate good psychometric properties of the H-SUS. Compared to the SUS, the H-SUS showed similar workload levels for questionnaire completion, higher levels of respondent motivation, but longer questionnaire completion time. Overall, the H-SUS is considered a promising alternative for the evaluation of perceived usability. Finally, QX can be considered a useful concept for identifying potential problems of psychometric instruments in a respondent-centred way, which may help improve the quality of future scales.

Using Tablets to Collect Breast Cancer Risk Information in an Underserved Population

The authors tested the use of tablets to collect breast cancer risk data and compared the approach against paper-based scantrons. The goal was to examine the usability of digital technology for data collection in underserved populations. A total of 340 individuals were randomized to answer a breast health survey via either tablets (170) or paper forms (170). Post questionnaire surveys were conducted to evaluate the usability of tablets. Outcomes included completion time, satisfaction, and the numbers and types of questions raised by patients during the survey. There was no significant difference in questionnaire completion time between two groups (12.5 vs. 12.1 minutes, p = 0.07). The tablet group was more satisfied with the experience (p &lt; 0.001). Low health literacy women spent more time on the questionnaire (p &lt; 0.05). Electronic devices can be utilized to collect breast cancer risk data in underserved, ethnically diverse populations. Future healthcare questionnaire applications should focus on accessibility improvements for these populations.

Usability Evaluation of KAI Access Application

The Government of Indonesia develops some distribution of transportation especially land transportation, i.e. train. This transportation only available in Java Island, but now available in Sumatra Island. The existence of this train was supported by the development of the Industry 4.0 which is train tickets purchased by Applications that can minimize cash payment, increasing the efficiency and effectiveness of customers. User ratings for KAI Access Application on Google Playstore is not good enough, there are still a lot of users who give complaints and bad ratings on this Application. From the data, there is a usability issue in this application. The purpose of this research is to evaluate KAI Access application with a usability approach in Sumatera Island which can be used as a basis for Increase performance of KAI Access application after comparing with Java Island users. This evaluation using a usability approach with the System Usability Scale (SUS) questionnaire, mapping complaints and task completion time for user performance. The result of this research are alternatives to improve this application such as redesign interface for home page and improvement of the database. Positive correlation is found between type of user (novice and expert) and origin of user (Java and Sumatra) where results show significant difference.

Using pens as an incentive for questionnaire return in an orthopaedic trial: an embedded randomised controlled retention trial

Background: We did a ‘study within a trial’ (SWAT), evaluating the effectiveness of the inclusion of a pen with a postal questionnaire, compared to no pen being included, on the retention rate in a large orthopaedic trial. Methods: The SWAT was embedded in the KReBS trial. The primary outcome was the proportion of 12-month questionnaires returned. Secondary outcomes were the proportion of questionnaires completed and time to questionnaire return. Binary data were analysed using logistic regression and time to return using Cox proportional hazards regression. Odds ratios (OR) and hazard ratios (HR) are presented, with associated 95% confidence intervals and p-values. Results: In total, 2306 participants were randomised into the SWAT. In the pen group, 1020/1146 (89.0%) of participants returned a questionnaire, compared to 982/1147 (85.6%) in the no pen group. The absolute difference in questionnaire return rate was 3.4% (95% CI: 0.7% to 6.1%; p=0.01). There were statistically significant differences in questionnaire return rate (OR 1.34; 95% CI: 1.04 to 1.72; p=0.02), questionnaire completion rate (OR 1.39; 95% CI: 1.10 to 1.76; p&lt;0.01) and time to questionnaire return (HR 1.16; 95% CI: 1.06 to 1.27; p&lt;0.01) favouring the pen group. Conclusion: This SWAT adds to the growing evidence base for whether pens are effective as an incentive for retention, and indicates their potential effectiveness. Registration: KReBS trial registered on 20 February 2019, ID ISRCTN87127065; SWAT registered on 1 April 2019, ID SWAT92.

Evaluation of reliability and validity of the patient health questionnaire-9 in patients with acne

Objective To evaluate the reliability and validity of the patient health questionnaire-9 (PHQ-9) in patients with acne. Methods A total of 300 patients with acne were enrolled from the Department of Dermatology, Affiliated Hospital of Southwestern Medical University from January 2017 to June 2018. Questionnaire survey was conducted among these patients to evaluate their depression status with the PHQ-9 and Hamilton depression scale (HAMD) , and the questionnaire completion time was recorded. The structured clinical interview (SCID) for American Diagnostic and Statistical Manual of Mental Disorders (4th Version) (DSM-Ⅳ) served as the for diagnosis of depression. Reliability analysis of the PHQ-9 was performed using the Cronbach′s α coefficient and one-week test-retest reliability. Factor analysis, Spearman correlation analysis and receiver operating characteristic (ROC) curve were used for validity analysis of the PHQ-9, and the screening validity was also analyzed in different subgroups. Results A total of 258 patients with acne completed all the survey, and 47 were diagnosed with depressive disorder according to the diagnostic gold standard SCID. An optimal cutoff score of 9 was determined for the diagnosis of depression using the PHQ-9, with a sensitivity of 95.7%, a specificity of 88.6%, and an area under the ROC curve (AUC) of 0.973 (95% CI: 0.956-0.990) . In different groups of people, the AUC ranged from 0.886 to 0.955. Validity analysis showed that the correlation coefficient between the total score of PHQ-9 and HAMD was 0.766, and the kappa value was 0.530. Factor analysis revealed that there were two common factors (cognitive-emotion factor and somatic symptom factor) , which explained 65.52% of the total variances. As reliability analysis showed, the Cronbach′s α coefficient of the PHQ-9 was 0.851, the correlation coefficients of the 9 items with the total scale score ranged from 0.550 to 0.709, the inter-item correlation coefficients ranged from 0.224 to 0.654, and the test-retest reliability value of the PHQ-9 was 0.824. The time to complete the PHQ-9 (4.9 ± 1.2 minutes) was significantly less than the time to complete HAMD (44.2 ± 5.9 minutes; t=108.787, P < 0.001) . Conclusions The PHQ-9 shows good reliability and validity for the diagnosis of depression in patients with acne, and can be used for preliminary screening of depression in patients with acne. Compared with the HAMD, the PHQ-9 has obvious advantages of being more timesaving and manpower-saving to make it easier to implement in clinic. Key words: Acne vulgaris; Depression; Questionnaires; Evaluation studies; Patient health questionnaire-9; Reliability; Validity

Patient preferences for a touch screen tablet-based asthma questionnaire

Objective: Assessment of asthma control and provision of asthma action plans are seldom performed in practice, partly due to limited time for physicians to ascertain required information. A patient-facing electronic asthma questionnaire could facilitate information collection. We sought to design a touch-tablet asthma questionnaire for use in the clinic waiting room and to describe patient preferences for the content of such a questionnaire. Methods: We created a questionnaire prototype based on best evidence and employed rapid-cycle design (semi-structured focus group testing; analysis; corresponding modifications and re-testing) with asthma patients aged ≥16 years. We analyzed transcripts using deductive and inductive content analysis. Quantitative measures included Likert-scale questions and questionnaire completion times. Results: There were 20 participants across five focus groups (15/20 (75%) female, age 49.1 ± 15.6 years). Content-related themes included: 1) comprehensibility (language) (how questionnaire language affected ease and accuracy of comprehension) and b) information collection (issues arising during information collection in the following identified subthemes: personal asthma symptoms and triggers; asthma control; asthma medications and contact information). Average questionnaire completion time was 11.7 ± 5.9 min. Summative Likert scale responses suggested high levels of question comprehension and confidence with responses. Conclusions: Our analysis provides novel insight about how best to formulate and present asthma-related content in an electronic questionnaire. Such questionnaires might facilitate quality improvement by improving efficiency of data collection, enabling better assessment of asthma control and medication adherence, and personalization of asthma action plans. Future studies should measure real-world uptake of such a questionnaire and impact on care.

How to Abbreviate Questionnaires and Avoid the Sins

Creating shortened versions of research tools is common and justified. Unfortunately, it is often performed without due methodological care and awareness of the consequences of such actions. Even though the errors committed during short form construction were collected by Smith and his collaborators in 2000, it did not distinctly affect the practice. The mistakes made by researchers still come down to two main faults: assuming the transferability of validity and reliability between the full and shortened versions, and lowering the validity and reliability requirements for short forms. These two problems manifest as 9 sins committed during the construction of short forms. This article intends to present procedures which prevent these mistakes and ensure creating possibly the most reliable short version of a research tool and assessing the costs of a selected shortening method. To this end, the work determined a priori the expected length of the tool, the benefit of reduced questionnaire completion time in relation to the cost of reliability loss. Also, it estimated overlapping variance of the full and short version and classification accuracy of the new, short version. Since there are many statistical techniques of questionnaire shortening, an additional effect of this article is a comparison of the efficiency of shortening by means of three various techniques. The results show similarity between the method based on factor loadings and Cronbach’s α method, and a slight advantage over the two of a method based on difficulty and discriminatory power in the IRT paradigm.

Medical end-of-life practices among Canadian physicians: a pilot study.

Medical end-of-life practices are hotly debated in Canada, and data from other countries are used to support arguments. The objective of this pilot study was twofold: to adapt and validate a questionnaire designed to measure the prevalence of these practices in Canada and the underlying decision-making process, and to assess the feasibility of a nationally representative study. In phase 1, questionnaires from previous studies were adapted to the Canadian context through consultations with a multidisciplinary committee and based on a scoping review. The modified questionnaire was validated through cognitive interviews with 14 physicians from medical specialties associated with a higher probability of being involved with dying patients recruited by means of snowball sampling. In phase 2, we selected a stratified random sample of 300 Canadian physicians in active practice from a national medical directory and used the modified tailored method design for mail and Web surveys. There were 4 criteria for success: modified questions are clearly understood; response patterns for sensitive questions are similar to those for other questions; respondents are comparable to the overall sampling frame; and mean questionnaire completion time is less than 20 minutes. Phase 1: main modifications to the questionnaire were related to documentation of all other medical practices (including practices intended to prolong life) and a question on the proportionality of drugs used. The final questionnaire contained 45 questions in a booklet style. Phase 2: of the 280 physicians with valid addresses, 87 (31.1%) returned the questionnaire; 11 of the 87 declined to participate, for a response rate of 27.1% (n = 76). Most respondents (64 [84%]) completed the mail questionnaire. All the criteria for success were met. It is feasible to study medical end-of-life practices, even for practices that are currently illegal, including the intentional use of lethal drugs. Results from this pilot study support conducting a large national study, but additional strategies would be necessary to improve the response rate.

Investigating respondent multitasking in web surveys using paradata

Computers play an important role in everyday multitasking. Within this context, we focus on respondent multitasking (RM) in web surveys. RM occurs when users engage in other activities while responding to a web survey questionnaire. The conceptual framework is built on existing literature on multitasking, integrating knowledge from both cognitive psychology and survey methodology. Our main contribution is a new approach for measuring RM in web surveys, which involves an innovative use of the different types of paradata defined as non-reactive electronic tracks concerning respondents' process of answering the web questionnaire. In addition to using questionnaire page completion time as a measure of RM, we introduce ‘focus-out’ events that indicate when respondents have left the window containing the web questionnaire (e.g., to chat, email, browse) and then returned. The approach was tested in an empirical study using a web survey on a student sample (n = 267). The results indicate that 60% of respondents have multitasked at least once. In addition, they reveal that item nonresponse as an indicator of response quality is associated with RM, while non-differentiation is not. Although this study confirms that a paradata-based approach is a feasible means of measuring RM, future research on this topic is warranted.

An Innovative Data Collection Method for Investigating Unresolved Pain After ED Discharge: A Pilot Study

Research focused on improving the outcome of patients with pain is an important aspect of emergency care, yet little information has been published to quantify pain severity, patient improvement, and return to independent function after ED discharge. Because of the paucity of available clinical data, this pilot study was designed to determine the acceptability and feasibility of using electronic data collection procedures. Specifically, we examined the willingness of discharged ED patients to use portable touch-screen computers as an alternative to traditional "paper and pencil" or telephone data collection methods to report postdischarge pain. Participants in this prospective pilot study all reported acute pain that was unresolved at the time of ED discharge. Descriptive data were self-entered on touch-screen computers at ED discharge and 7 days later in subjects' homes. Participants completed 4 Patient-Reported Outcomes Measurement Information System (PROMIS Network, Silver Spring, MD), questionnaires and the computer acceptability scale (CAS), using the Assessment Center platform program (National Institute of Health, Bethesda, MD) for instrument administration, data storage, and information retrieval. Determinants of study success were willingness of the participants to use touch-screen computers, questionnaire completion time, and completeness of the data. Participants (N = 25) used touch-screen computers to complete 18 questions at the time of ED discharge. The mean completion time was 3.54 (± 1.13) minutes. Participants averaged 5.83 (± 2.00) minutes to complete the follow-up questionnaires. Ninety-two percent of subjects reported that the touch-screen computer was easy to use. We encountered no issues with data management using the Assessment Center platform. Touch-screen computers are a feasible and acceptable approach to collecting information about how patients self-manage unresolved pain after discharge from an emergency department. This methodology offers an alternative to traditional data collection methods. These data can inform researchers as they design future studies and assist emergency nurses who are responsible for planning quality improvement initiatives.