The present study investigates the factors of “Weekday sleep debt (WSD)” by comparing activity data collected from persons with and without WSD. Since it has been reported that the amount of sleep debt as well the difference between the social clock and the biological clock is associated with WSD, specifying the factors of WSD other than chronotype may contribute to sleep debt prevention. We recruited 324 healthy male employees working at the same company and collected their 1-week wrist actigraphy data and answers to questionnaires. Because 106 participants were excluded due to measurement failure of the actigraphy data, the remaining 218 participants were included in the analysis. All participants were classified into WSD or non-WSD groups, in which persons had WDS if the difference between their weekend sleep duration and the mean weekday sleep duration was more than 120 min. We evaluated multiple measurements derived from the collected actigraphy data and trained a classifier that predicts the presence of WSD using these measurements. A support vector machine (SVM) was adopted as the classifier. In addition, to evaluate the contribution of each indicator to WSD, permutation feature importance was calculated based on the trained classifier. Our analysis results showed significant importance of the following three out of the tested 32 factors: (1) WSD was significantly related to persons with evening tendency. (2) Daily activity rhythms and sleep were less stable in the WSD group than in the non-WSD group. (3) A specific day of the week had the highest importance in our data, suggesting that work habit contributes to WSD. These findings indicate some WSD factors: evening chronotype, instability of the daily activity rhythm, and differences in work habits on the specific day of the week. Thus, it is necessary to evaluate the rhythms of diurnal activities as well as sleep conditions to identify the WSD factors. In particular, the diurnal activity rhythm influences WSD. It is suggested that proper management of activity rhythm may contribute to the prevention of sleep debt.