To develop and evaluate the reliability and validity of a new observational Drinking Task Assessment (DTA) designed to assess quality of movement in task performance after stroke. Reliability and validity. The DTA measures movement time and movement quality (smoothness, trunk, shoulder, elbow, and grasp movements) on a 4-level ordinal scale. Thirty participants with chronic stroke were assessed independently by 2 therapists. Intra-class correlation (ICC), standard error of measurement (SEM) and minimal real difference (MRD), weighted kappa, percentage of agreement, and Svensson method were used for reliability assessment. Motion capture-based kinematics and established clinical scales were used to evaluate validity. The absolute SEM and MRD for movement time were 0.4 and 1 s (11%), respectively. The ICC (≥ 0.93) and weighted kappa (0.71-1.0) showed good to excellent agreement for intra- and inter-rater reliability. DTA showed strong correlations with Fugl-Meyer Assessment (0.74), Action Research Arm Test (0.93), and kinematic measures of smoothness (0.93), trunk displacement (0.91), elbow extension (0.73), and shoulder movements (0.56), indicating good construct validity. The new DTA proved to be a reliable and valid tool for assessment of movement quality during task performance after stroke.