Number Of Unique Designs Research Articles

The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)

BackgroundThe Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT.MethodsWe randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots.ResultsWe included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83).ConclusionsThe automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores.

The validity and reliability of the digital Ruff Figural Fluency Test (RFFT) in the general adult population

AbstractBackgroundThe Ruff Figural Fluency Test (RFFT) is a valid pencil‐and‐paper test for evaluating cognitive functioning in the general population. The assessment of the pencil‐and‐paper‐RFFT is time‐consuming and requires training of raters. We developed a digital‐RFFT which can be filled in by people independently on an Ipad, after which RFFT patterns are automatically processed and stored. We aimed to investigate the validity and reliability of the digital‐RFFT in the general population.MethodStratified randomization (based on age, gender, and education) was used to allocate participants either to the digital‐RFFT or the pencil‐and‐paper‐RFFT. After the first test, the other test was repeated (cross‐over). The number of unique designs and perseverative errors of the pencil‐and‐paper‐RFFT and digital‐RFFT were assessed by an computerized algorithm (automatic) and by two independent trained human raters (criterion standard). For the statistical analysis, we used Intraclass correlation coefficients(ICC), sensitivity, specificity, %agreement, weighted Kappa, and Bland Altman plots, including limits‐of‐agreement(LoA).ResultsThe study population comprised 96 participants (58% women; mean(SD) age 40(15)years). The mean(SD) number of unique designs was 84.2 (26.0) (automatic assessment) and 85.3 (26.2) (human assessment) for the digital‐RFFT and 91.3 (22.7) for the pencil‐and‐paper‐RFFT. The median number(IQR) of perseverative errors was 4(2‐7) (automatic assessment) and 4.5(2‐8) (human assessment) for the digital‐RFFT and 4.5(2‐7.5) for the pencil‐and‐paper‐RFFT. The percentage agreement (kappa) between the two human raters for the digital‐RFFT and pencil‐and‐paper‐RFFT was 0.94(0.90) and 0.94(0.81), respectively. ICC between human assessment and automatic assessment of the digital‐RFFT (unique designs) was 0.988(95%CI:0.981‐0.992); sensitivity and specificity was 0.98 and 0.96 respectively; mean difference(95%CI):‐1.12(‐1.92,‐0.33); 95%Limits‐of‐Agreement ‐8.75,6.51). The ICC between digital‐RFFT and pencil‐and‐paper‐RFFT was 0.53(95%CI 0.37,0.67). When starting with the paper‐and‐pencil‐RFFT, this ICC was 0.73 (95%CI:0.34,0.87). The mean difference (unique designs) between the digital‐RFFT and pencil‐and‐paper‐RFFT was ‐7.09 (95%CI ‐11.8 to ‐2.38 (95%limits‐of‐agreement: ‐52.12, 37.94). This was due to unclear instructions (i.e. complex examples) that were given on the Ipad (digital‐RFFT) and learning effect.ConclusionsAlthough small and systematic differences between human assessment and automatic assessment of the digital‐RFFT were found, we conclude that the digital‐RFFT is a valid test to assess cognitive functioning among the general population if instructions of the digital‐RFFT are improved.

Development of normative data for the Five Point Test using the cognitive reliability and time to failure rate theory

In this study, several theoretically based curve fitting nonlinear models for analyzing the Five Point Test (FPT), a nonverbal fluency test, were examined. One thousand two hundred and one participants from the general population of Germany and the USA completed the FPT. The test scores were analyzed using three process indexes; the number of unique designs, strategy, and repetitions. Participants were stratified by age and level of education, and the resulting data were examined using best-fit nonlinear distribution equations. The study shows that the Weibull Growth three-factor curve fitting model provides equivalent fit as four-factor quadratic or linear curve fitting models for estimates continue normative data the Reliability and Time to Failure Theory of Cognitive Functions. Namely, that neurocognitive abilities remain stable during young adulthood and after reaching a threshold, starts to linearly or exponentially decline. The study also shows that education differentially impacts the asymptote, threshold, and decline rate of the FPT indexes confirming the importance of isolating specific cognitive domains within neuropsychological tests as these indexes might be related to the activation of particular cell assemblies, which in turn are differentially impacted by aging.

A-241 Five Point Test: Reliability, Validity, and Test Performance in Cognitively Intact Adults

Abstract Objective Reliability and validity evidence related to the Five Point Test (FPT) scores is severely limited. The primary purpose of this study was to examine psychometric evidence related to two commonly used FPT scores (number of unique designs (UD), percentage of repetitions (PR)) using one-week test–retest reliability, correlations with demographic and neuropsychological variables, and convergent validity in line with a regression-based explanation-focused view of validity. Methods The sample consisted of 86 cognitively intact, non-depressed adult men and women ages 21–82 years (M = 52.7, SD = 17.7) with 7–21 years of education (M = 14.2, SD = 3.13) recruited from the general community and tested individually. Results UD ranged from 8–60 (M = 35.4) and PR ranged from 0–45% (M = 6.9%). Test–retest coefficients were .83 for UD but only .43 for PR. Age was significantly correlated with UD (r = −.59) and PR (r = .23). Education was significantly correlated with UD (r = .26) but not PR (r = −.10). There were no gender differences. UD showed significant bivariate correlations with WAIS-III Block Design, Trail-Making Test (TMT) A, TMT-B, Bicycle Drawing Test, and FAS Verbal Fluency but, together in a regression, only age and TMT-B remained significant. PR scores did not correlate significantly with any neuropsychological variables. Conclusion UD showed strong test–retest reliability. UD performance tends to be poorer with older age and less education. The meaning and interpretation of UD performance using a regression-based explanation-focused view of validity will be discussed. PR reliability is poor even over a short interval and attenuates subsequent statistical findings. Use of PR is not recommended in research or practice.

Agreement between Computerized and Human Assessment of Performance on the Ruff Figural Fluency Test

The Ruff Figural Fluency Test (RFFT) is a sensitive test for nonverbal fluency suitable for all age groups. However, assessment of performance on the RFFT is time-consuming and may be affected by interrater differences. Therefore, we developed computer software specifically designed to analyze performance on the RFFT by automated pattern recognition. The aim of this study was to compare assessment by the new software with conventional assessment by human raters. The software was developed using data from the Lifelines Cohort Study and validated in an independent cohort of the Prevention of Renal and Vascular End Stage Disease (PREVEND) study. The total study population included 1,761 persons: 54% men; mean age (SD), 58 (10) years. All RFFT protocols were assessed by the new software and two independent human raters (criterion standard). The mean number of unique designs (SD) was 81 (29) and the median number of perseverative errors (interquartile range) was 9 (4 to 16). The intraclass correlation coefficient (ICC) between the computerized and human assessment was 0.994 (95%CI, 0.988 to 0.996; p<0.001) and 0.991 (95%CI, 0.990 to 0.991; p<0.001) for the number of unique designs and perseverative errors, respectively. The mean difference (SD) between the computerized and human assessment was -1.42 (2.78) and +0.02 (1.94) points for the number of unique designs and perseverative errors, respectively. This was comparable to the agreement between two independent human assessments: ICC, 0.995 (0.994 to 0.995; p<0.001) and 0.985 (0.982 to 0.988; p<0.001), and mean difference (SD), -0.44 (2.98) and +0.56 (2.36) points for the number of unique designs and perseverative errors, respectively. We conclude that the agreement between the computerized and human assessment was very high and comparable to the agreement between two independent human assessments. Therefore, the software is an accurate tool for the assessment of performance on the RFFT.

Longitudinal study of performance on the Ruff Figural Fluency Test in persons aged 35 years or older.

The Ruff Figural Fluency Test (RFFT) is a cognitive test to measure executive function. Longitudinal studies have shown that repeated testing improves performance on the RFFT. Such a practice effect may hinder the interpretation of test results in a clinical setting. Therefore, we investigated the longitudinal performance on the RFFT in persons aged 35–82 years. Performance on the RFFT was measured three times over an average follow-up period of six years in 2,515 participants of the Prevention of REnal and Vascular ENd-stage Disease (PREVEND) study in Groningen, the Netherlands: 53% men; mean age (SD), 53 (10) years. The effect of consecutive measurements on performance on the RFFT was investigated with linear multilevel regression models that also included age, gender, educational level and the interaction term consecutive measurement number x age as independent variables. It was found that the mean (SD) number of unique designs on the RFFT increased from 73 (26) at the first measurement to 79 (27) at the second measurement and to 83 (26) at the third measurement (p<0.001). However, the increase per consecutive measurement number was negatively associated with age and decreased with 0.23 per one-year increment of age (p<0.001). The increase per consecutive measurement number was not dependent on educational level. Similar results were found for the median (IQR) number of perseverative errors which showed a small but statistically significant increase with repeating testing: 7 (3–13) at the first measurement, 7 (4–14) at the second measurement and 8 (4–15) at the third measurement (p trend = 0.002). In conclusion, the performance on the RFFT improved by repeating the test over an average follow-up period of three to six years. This practice effect was the largest in young adults and not dependent on educational level.

Reference Data for the Ruff Figural Fluency Test Stratified by Age and Educational Level

The Ruff Figural Fluency Test (RFFT) was developed to avoid the difficulties that were encountered in earlier tests of figural fluency. Although the test characteristics of the RFFT seem to be good and it is a valuable addition to neuropsychological assessments, reference data are still scarce. To this aim, we required 2,404 community dwelling persons in Groningen, the Netherlands to perform the RFFT. All 1,651 persons with a complete RFFT and known educational level formed the reference sample. Their age ranged from 35 to 82 years and their educational level from primary school to university grade. Ninety-six percent of the persons were of Western European descent. All tests were analyzed by two independent examiners and subsequently three measures were calculated: number of unique designs, number of perseverative errors and error ratio. The main finding was that performance on the RFFT was dependent on age and educational level. This was not only observed in older persons but also in young and middle-aged persons. Reference data for the three RFFT measures are presented in groups of five years of age ranging from 35–39 years to 75 years or older.

On the Duration of Spatial Fluency Measures

Various measures of spatial fluency have been developed and have been shown to be sensitive to right frontal lobe dysfunction. Patients with diffuse cerebral injuries (traumatic brain injury) also show impaired performance. Administration times for these tests range from three to five minutes. It is well known that longer tests provide more reliable data. in the present study, the base rates of unique designs and perseverative errors on the Five-Point Test were examined minute-by-minute for ten minutes, in a sample of healthy young adults (n = 80). Contrasts between each minute showed significant decreases in number of unique designs to the ninth minute (significant at p <. 001). Contrasts between each minute revealed significant increases (significant at p <. 001) in percentage of perseverative errors to the eighth minute. Data demonstrating the progressive decrease of unique designs and progressive increase of perseverative errors as a function of time have important implications for clinical practice. Optimal administration time is considered in the context of clinicians' objectives.

Children's figural fluency performance: Development of strategy use

To this point, the focus in evaluating measures of figural fluency, including the Ruff Figural Fluency Test (RFFT), has been on the number of unique designs generated and the number of repetitions or perseverations. In addition to these quantitative measures, our study assesses the qualitative aspects of figural fluency production in determining the extent to which children use strategies on the RFFT. Accordingly, two main strategies were opera‐tionalized, and design complexity (number of lines used for each design) was analyzed. It was postulated that age would play a significant role. The 87 children tested were, therefore, selected across four age groupings: Grades 1 and 2, Grades 3 and 4, Grades 5 and 6, and Grades 7 and 8. Each child was individually administered all five parts of the RFFT. The results indicated that older children used strategies to complete the test, whereas young children did not, and that design complexity did not discriminate children according to age or grade.