Abstract
Wrist-worn inertial measurement units have emerged as a promising technology to passively capture dietary intake data. State-of-the-art approaches use deep neural networks to process the collected inertial data and detect characteristic hand movements associated with intake gestures. In order to clarify the effects of data preprocessing, sensor modalities, and sensor positions, we collected and labeled inertial data from wrist-worn accelerometers and gyroscopes on both hands of 100 participants in a semi-controlled setting. The method included data preprocessing and data segmentation, followed by a two-stage approach. In Stage 1, we estimated the probability of each inertial data frame being intake or non-intake, benchmarking different deep learning models and architectures. Based on the probabilities estimated in Stage 1, we detected the intake gestures in Stage 2 and calculated the F1 score for each model. Results indicate that top model performance was achieved by a CNN-LSTM with earliest sensor data fusion through a dedicated CNN layer and a target matching technique (F 1 = .778). As for data preprocessing, results show that applying a consecutive combination of mirroring, removing gravity effect, and standardization was beneficial for model performance, while smoothing had adverse effects. We further investigate the effectiveness of using different combinations of sensor modalities (i.e., accelerometer and/or gyroscope) and sensor positions (i.e., dominant intake hand and/or non-dominant intake hand).
Highlights
Advances in mobile sensor technologies have enabled novel forms of dietary assessment
While dietary assessment was traditionally carried out exclusively using active methods for capturing food intake based on human effort to collect data (e.g., 24-hr recalls, food records), passive capture methods aim to reduce burden on individuals associated with collecting dietary data by using a range of different sensor technologies
(2) Proposed Model and Benchmarking: We propose a new model that achieved better performance (F1 = .778) compared to current state-of-the-art deep learning models for detecting intake gestures based on inertial data, using our novel large-scale dataset
Summary
Advances in mobile sensor technologies have enabled novel forms of dietary assessment. While dietary assessment was traditionally carried out exclusively using active methods for capturing food intake based on human effort to collect data (e.g., 24-hr recalls, food records), passive capture methods aim to reduce burden on individuals associated with collecting dietary data by using a range of different sensor technologies (e.g., inertial measurement units, microphones, The associate editor coordinating the review of this manuscript and approving it for publication was Gang Mei. and video cameras). Sensor technologies have the potential of complementing active capture methods for quantifying food intake [1] (e.g., by verifying intake activities, prompting human capture). The wrist-worn Inertial Measurement Unit (IMU) has emerged as a promising technology for sensor-based passive capture of food intake [2]–[4]. Triaxial accelerometers in IMUs measure changes in speed and
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
