Exploring the Performance of PySpark and Scikit-Learn Libraries in Developing Fall Detection Systems

Abstract

Falls pose a significant risk for the elderly population, often resulting in serious injuries and decreased quality of life. Accurate and effective fall detection systems can play an important role in reducing these risks. This study conducts a comparative analysis of the performance of PySpark and Scikit-Learn libraries in the development of fall detection models. PySpark offers a distributed computing environment specifically designed for big data processing, while Scikit-Learn is a Python-based library widely used for data analysis and machine learning. Five popular machine learning algorithms, including logistic regression, gradient boosting classifier, random forest, support vector machine, and Decision Tree, were used to build fall detection models using both PySpark and Scikit-Learn. Models were evaluated using a comprehensive set of measurements including accuracy, sensitivity, specificity, and confusion matrix. In addition to the five popular machine learning algorithms included in the study, 26 different features were extracted from the Sisfall dataset, one of the most comprehensive fall and daily life activity datasets, in five main categories: basic statistical features, frequency domain features, time series features, motion features, and relational features. These features were incorporated into fall detection models to improve their ability to accurately identify falls. The findings reveal that both PySpark and Scikit-Learn provide strong and effective results in fall detection. Logistic regression emerged as the best performing algorithm in both libraries, achieving the highest accuracy rates. However, PySpark exhibited slightly longer training times compared to Scikit-Learn, which performed better in testing. Ultimately, this study will benefit researchers and practitioners involved in the development of robust and effective fall detection systems, thereby enabling improved safety and well-being for the elderly population. Additionally, it contributes to the literature by presenting a comprehensive feature extraction methodology for fall detection, which has not been explored extensively in previous research works.