Fine-grained Time Research Articles

Facial Expression Recognition for Probing Students’ Emotional Engagement in Science Learning

AbstractStudent engagement in science classroom is an essential element for delivering effective instruction. However, the popular method for measuring students’ emotional learning engagement (ELE) relies on self-reporting, which has been criticized for possible bias and lacking fine-grained time solution needed to track the effects of short-term learning interactions. Recent research suggests that students’ facial expressions may serve as an external representation of their emotions in learning. Accordingly, this study proposes a machine learning method to efficiently measure students’ ELE in real classroom. Specifically, a facial expression recognition system based on a multiscale perception network (MP-FERS) was developed by combining the pleasure-displeasure, arousal-nonarousal, and dominance-submissiveness (PAD) emotion models. Data were collected from videos of six physics lessons with 108 students. Meanwhile, students’ academic records and self-reported learning engagement were also collected. The results show that students’ ELE measured by MP-FERS was a significant predictor of academic achievement and a better indicator of true learning status than self-reported ELE. Furthermore, MP-FERS can provide fine-grained time resolution on tracking the changes in students’ ELE in response to different teaching environments such as teacher-centered or student-centered classroom activities. The results of this study demonstrate the validity and utility of MP-FERS in studying students’ emotional learning engagement.

Predicting Surgical Outcomes Using Wearables

Accurate prediction of readmission and severe complication risks during the pre-operative stage is crucial for clinical decision-making. In addition to traditional static clinical data, the activity and physiological data collected by commercial wearable devices such as Fitbit wristbands provide valuable complementary information. However, data from wearables are often incomplete and noisy, presenting challenges in extracting clinically meaningful features from fine-grained time series data. In this paper, we propose a pipeline to develop predictive models using pre-operative wearable data to predict surgical outcomes in patients undergoing pancreatic surgeries. The core of this pipeline is a multi-level feature engineering framework designed to extract high-level features from wearable data, which are then combined with static clinical characteristics to form the input for the predictive model. In a clinical study involving 61 patients, our approach improved the Area Under the Receiver Operating Characteristic (AUROC) by 44% compared to the existing risk calculator employed in clinical practice.

RimSense

Smart eyewear's interaction mode has attracted significant research attention. While most commercial devices have adopted touch panels situated on the temple front of eyeglasses for interaction, this paper identifies a drawback stemming from the unparalleled plane between the touch panel and the display, which disrupts the direct mapping between gestures and the manipulated objects on display. Therefore, this paper proposes RimSense, a proof-of-concept design for smart eyewear, to introduce an alternative realm for interaction - touch gestures on eyewear rim. RimSense leverages piezoelectric (PZT) transducers to convert the eyeglass rim into a touch-sensitive surface. When users touch the rim, the alteration in the eyeglass's structural signal manifests its effect into a channel frequency response (CFR). This allows RimSense to recognize the executed touch gestures based on the collected CFR patterns. Technically, we employ a buffered chirp as the probe signal to fulfil the sensing granularity and noise resistance requirements. Additionally, we present a deep learning-based gesture recognition framework tailored for fine-grained time sequence prediction and further integrated with a Finite-State Machine (FSM) algorithm for event-level prediction to suit the interaction experience for gestures of varying durations. We implement a functional eyewear prototype with two commercial PZT transducers. RimSense can recognize eight touch gestures on the eyeglass rim and estimate gesture durations simultaneously, allowing gestures of varying lengths to serve as distinct inputs. We evaluate the performance of RimSense on 30 subjects and show that it can sense eight gestures and an additional negative class with an F1-score of 0.95 and a relative duration estimation error of 11%. We further make the system work in real-time and conduct a user study on 14 subjects to assess the practicability of RimSense through interactions with two demo applications. The user study demonstrates RimSense's good performance, high usability, learnability and enjoyability. Additionally, we conduct interviews with the subjects, and their comments provide valuable insight for future eyewear design.

Real-time bio-layer interferometry ubiquitination assays as alternatives to western blotting

Ubiquitination is a crucial cellular pathway enabling normal cellular functions. Abnormalities in the ubiquitination process can lead to cellular dysfunction and cause a range of diseases. Efforts to screen and develop small molecule inhibitors targeting portions of the ubiquitination cascade require rapid and robust methods for detecting ubiquitination. Enormous efforts have been made in the field to detect ubiquitination using various techniques including fluorescence, spectrophotometry, chemiluminescence, NMR, and radioactive tracers. The most common method to detect ubiquitination is western blotting. However, western blotting is time-consuming and difficult to use when seeking fine-grained time course experiments. Here we present the use of bio-layer interferometry to rapidly assay ubiquitination in real-time. An E3 ligase auto-ubiquitination system and a substrate ubiquitination assay have been applied as tests for the newly developed assay. The developed BLI ubiquitination assay provides one-second time resolution and detects the formation of polyubiquitin chains directly on a biosensor-bound target. Results are returned instantaneously, and reagent concentrations are identical to those used by traditional western blot-based ubiquitination assays. The developed BLI ubiquitination assay is a viable alternative to traditional western blot assays to detect ubiquitination in a rapid real-time manner.

Fine-grained Network Traffic Prediction from Coarse Data

ICT systems provide detailed information on computer network traffic. However, due to storage limitations, some of the information on past traffic is often only retained in an aggregated form. In this paper we show that Linear Gaussian State Space Models yield simple yet effective methods to make predictions based on time series at different aggregation levels. The models link coarse-grained and fine-grained time series to a single model that is able to provide fine-grained predictions. Our numerical experiments show up to 3.7 times improvement in expected mean absolute forecast error when forecasts are made using, instead of ignoring, additional coarse-grained observations. The forecasts are obtained in a Bayesian formulation of the model, which allows for provisioning of a traffic prediction service with highly informative priors obtained from coarse-grained historical data.

A time slice analysis of dentistry students' visual search strategies and pupil dilation during diagnosing radiographs.

Diagnosing orthopantomograms (OPTs: panoramic radiographs) is an essential skill dentistry students acquire during university training. While prior research described experts' visual search behavior in radiology as global-to-focal for chest radiographs and mammography, generalizability to a hybrid search task in OPTs (i.e., searching for multiple, diverse anomalies) remains unclear. Addressing this gap, this study investigated visual search of N = 107 dentistry students while they were diagnosing anomalies in OPTs. Following a global-to-focal expert model, we hypothesized that students would use many, short fixations representing global search in earlier stages, and few, long fixations representing focal search in later stages. Furthermore, pupil dilation and mean fixation duration served as cognitive load measures. We hypothesized that later stages would be characterized by elaboration and a reflective search strategy, leading to higher cognitive load being associated with higher diagnostic performance in late compared to earlier stages. In line with the first hypothesis, students' visual search comprised of a three-stage process that grew increasingly focal in terms of the number of fixations and anomalies fixated. Contrary to the second hypothesis, mean fixation duration during anomaly fixations was positively associated with diagnostic performance across all stages. As OPTs greatly varied in how difficult it was to identify the anomalies contained therein, OPTs with above-average difficulty were sampled for exploratory analysis. Pupil dilation predicted diagnostic performance for difficult OPTs, possibly capturing elaborative cognitive processes and cognitive load compared to mean fixation duration. A visual analysis of fine-grained time slices indicated large cognitive load differences towards the end of trials, showcasing a richness-resolution-trade-off in data sampling crucial for future studies using time-slicing of eye tracking data.

Large-Scale Estimation and Analysis of Web Users' Mood from Web Search Query and Mobile Sensor Data.

The ability to estimate the current mood states of web users has considerable potential for realizing user-centric opportune services in pervasive computing. However, it is difficult to determine the data type used for such estimation and collect the ground truth of such mood states. Therefore, we built a model to estimate the mood states from search-query data in an easy-to-collect and non-invasive manner. Then, we built a model to estimate mood states from mobile sensor data as another estimation model and supplemented its output to the ground-truth label of the model estimated from search queries. This novel two-step model building contributed to boosting the performance of estimating the mood states of web users. Our system was also deployed in the commercial stack, and large-scale data analysis with >11 million users was conducted. We proposed a nationwide mood score, which bundles the mood values of users across the country. It shows the daily and weekly rhythm of people's moods and explains the ups and downs of moods during the COVID-19 pandemic, which is inversely synchronized to the number of new COVID-19 cases. It detects big news that simultaneously affects the mood states of many users, even under fine-grained time resolution, such as the order of hours. In addition, we identified a certain class of advertisements that indicated a clear tendency in the mood of the users who clicked such advertisements.

Quantum compiling with a variational instruction set for accurate and fast quantum computing

The quantum instruction set (QIS) is defined as the quantum gates that are physically realizable by controlling the qubits in quantum hardware. Compiling quantum circuits into the product of the gates in a properly defined QIS is a fundamental step in quantum computing. We here propose the quantum variational instruction set (QuVIS) formed by flexibly designed multi-qubit gates for higher speed and accuracy of quantum computing. The controlling of qubits for realizing the gates in a QuVIS is variationally achieved using the fine-grained time optimization algorithm. Significant reductions in both the error accumulation and time cost are demonstrated in realizing the swaps of multiple qubits and quantum Fourier transformations, compared with the compiling by a standard QIS such as the quantum microinstruction set (QuMIS, formed by several one- and two-qubit gates including one-qubit rotations and controlled-NOT gates). With the same requirement on quantum hardware, the time cost for QuVIS is reduced to less than one half of that for QuMIS. Simultaneously, the error is suppressed algebraically as the depth of the compiled circuit is reduced. As a general compiling approach with high flexibility and efficiency, QuVIS can be defined for different quantum circuits and be adapted to the quantum hardware with different interactions.

The punishment is the reward – why elite Chinese students take supplementary Chinese literacy tutoring despite viewing it as ineffective

Abstract Reward and punishment are basic concepts in psychology. To survive better, humans and animals often need to maximize rewards and minimize punishments and threats. The human brain makes judgments about rewards and punishments based on distinguishing among complex environmental factors and makes decisions about future actions accordingly. Chinese students have spent more than a decade of primary and secondary school education with purposeful, fine-grained time management learning with the goal of success on the college entrance exam (Gaokao). The relationship between rewards and punishments for literacy tutoring at the primary and secondary school levels and the decision-making behavior of students and parents is therefore a question worth exploring. The purpose of this paper is to explore the decision-making mechanisms of elite Chinese students’ participation in or withdrawal from literacy tutoring in Chinese and the factors behind the reward and punishment mechanisms through grounded method analysis. The results show that most elite students and their parents do not believe that language literacy tutoring will significantly improve their chances for better college entrance exam scores, but there is a complex relationship between their decision-making behavior in tutoring and the reward and punishment mechanisms. Literacy improvement, emotional values, parent-child support obligations and even face-saving culture all contribute to students’ and parents’ decision-making behaviors. Choosing tutoring that is not seen as likely to improve test scores is a common phenomenon, which suggests that the punishment itself provides the reward that drives decisions to pursue literacy tutoring.

Optimization for Metro Operation Scheme of Suburban Lines: A New Method for Dealing with the Imbalanced Passenger Flow

The development of some central cities tends to be saturated, so some mega cities try to adopt the express and local mode of the metro. Suburban lines with imbalanced passenger flow between stations will lead to extremely crowded passenger flow in some stations, while scarce passenger flow in other stations. To balance the train’s carrying capacity with the imbalanced passenger flow, this study explores the collaborative optimization of train operation under the new mode of express and local. First, a time-stamped data cropping strategy based on fine-grained time zones as the grid index is developed, which is used to deeply explore the OD spatiotemporal representation of passenger flow correlation mapping mechanism based on multisource data; then, the calculation model of the proportion of trains matching with the OD temporal and spatial characteristics of suburban passenger flow are constructed, and an efficient solution algorithm is developed to solve the problem of interest. Finally, a set of numerical experiments with operation data from Shanghai Metro Line 16 are conducted to verify the performance and effectiveness of the proposed model and algorithm. The experimental results show that the proposed approach can effectively realize the collaborative optimization of passenger OD prediction, train proportion, stop scheme, and travel time, so as to provide decision-making support and method guidance for the optimization of metro organizations in megacities.

Role of Pitch in Toddler Looking to New and Given Referents in American English

ABSTRACT This study examined how toddler looking to a discourse referent is mediated by the information status of the referent and the pitch contour of the referring expression. Eighteen-month-olds saw a short discourse of three sets of images with the proportion of looking time to a target analyzed during the final image. At test, the information status of the referent was either new or given and the referring expression was presented with one of three pitch contours (flat f0, monotonal (~H*), or bitonal (~L+H*)). In Experiment 1, toddlers looked reliably longer to a target referent when it was either new to the discourse or carried a non-flat pitch contour. In Experiment 2, the referring expression was removed to observe effects of information status alone on looking to a target referent. Toddlers looked significantly longer to a target when it was new versus given. More fine-grained time course analyses of eye movements revealed differences in the speed and duration of fixation to a target. Overall, the experiments show that discourse reference in toddlers is mediated by the presence of newness and pitch contours, even in the case of given information.

The time course of lexical and sublexical phonological activation in Chinese written production

The time course of sublexical processing in written production is scarcely investigated due to transparent sound-to-spelling correspondence in most alphabetical languages. By taking advantage of the opaque orthography of Chinese, we were able to dissociatively manipulate sublexical relatedness and lexical relatedness in a picture-word interference task. The ERP technique was also used to provide a fine-grained time course of sublexical processing. Participants were asked to handwrite picture names while ignoring distractors overlain on the pictures. Distractors were phonologically related to the picture names (lexically related), phonologically related to the phonetic radicals of picture names (sublexically related), or unrelated to picture names. ERP results revealed that sublexical relatedness affected ERPs in the time window of 370–470 ms after picture onset, which was later than the early time window (200–260 ms) of the effect of lexical relatedness. We suggest that the sublexical phonological activation happens in the stages of sublexical POC procedure and/or the graphemic output buffer in written production. The relatively later time window of the sublexical effect is consistent with the later loci of the sublexical processes in the model of written production.

Dual-grained directional representation for infectious disease case prediction

The uncertain infection transmission causes challenges in accurate disease prediction. Numerous methods have been proposed to capture the temporal pictures from past observations within equal time intervals, which are called single-grained time series. However, these methods are not suitable for capturing uncertain temporal dynamics from infectious disease time series, since the infectious diseases may propagate in the incubation period. To address this issue, this paper proposes a Dual-Grained Directional Representation (DGDR) to generate predictions, via consolidating the representations of an equal-grained time series and several fine-grained time series. Firstly, the proposed DGDR learns a transformed segmentation into three kinds of representations. And then those representations from both equal-grained data and fine-grained data are temporally consolidated to connect with outputs. Extensive experiments on two real infectious disease datasets are done to validate the proposed DGDR. Compared with the other twelve methods, MAE value is decreased by 31.5%, RMSE value is decreased by 29.9%, and R2 value is improved by 87.6%.

Maximum Flow Routing Strategy With Dynamic Link Allocation for Space Information Networks Under Transceiver Constraints

In this paper, we investigate the maximum flow routing strategy with dynamic link allocation under a constraint on the number of transceivers for space information networks (SINs). Specifically, the time-expanded graph (TEG) is exploited to characterize the dynamic topology of SINs. Furthermore, although there exist multiple feasible links for SINs, only a limited number of them can be actually established due to the constraint on the number of transceivers. Traditionally the established link is fixed within one time interval in the TEG. In order to fully exploit the resource of multiple feasible links, we divide each time interval in the TEG into multiple fine-grained time periods and design the maximum flow routing strategy by jointly optimizing both the fine-grained time period duration and the link allocation as well as the amount of data transmitted on each transmission link and the amount of data stored in each caching link. This problem can be formulated as a mixed-integer quadratic program (MIQP), which is difficult to solve. To overcome this difficulty, we transform the MIQP into an equivalent mixed-integer linear program (MILP), which can be effectively solved by existing methods. Simulation results show that the proposed dynamic link allocation strategy can significantly outperform the fixed link allocation strategy within each time interval.

Head-mounted mobile eye-tracking in the domestic dog: A new method.

Humans rely on dogs for countless tasks, ranging from companionship to highly specialized detection work. In their daily lives, dogs must navigate a human-built visual world, yet comparatively little is known about what dogs visually attend to as they move through their environment. Real-world eye-tracking, or head-mounted eye-tracking, allows participants to freely move through their environment, providing more naturalistic results about visual attention while interacting with objects and agents. In dogs, real-world eye-tracking has the potential to inform our understanding of cross-species cognitive abilities as well as working dog training; however, a robust and easily deployed head-mounted eye-tracking method for dogs has not previously been developed and tested. We present a novel method for real-world eye-tracking in dogs, using a simple head-mounted mobile apparatus mounted onto goggles designed for dogs. This new method, adapted from systems that are widely used in humans, allows for eye-tracking during more naturalistic behaviors, namely walking around and interacting with real-world stimuli, as well as reduced training time as compared to traditional stationary eye-tracking methods. We found that while completing a simple forced-choice treat finding task, dogs look primarily to the treat, and we demonstrated the accuracy of this method using alternative gaze-tracking methods. Additionally, eye-tracking revealed more fine-grained time course information and individual differences in looking patterns.

Predicting Post-Operative Complications with Wearables

Post-operative complications and hospital readmission are of great concern to surgical patients and health care providers. Wearable devices such as Fitbit wristbands enable long-term and non-intrusive monitoring of patients outside clinical environments. To build accurate predictive models based on wearable data, however, requires effective feature engineering to extract high-level features from time series data collected by the wearable sensors. This paper presents a pipeline for developing clinical predictive models based on wearable sensors. The core of the pipeline is a multi-level feature engineering framework for extracting high-level features from fine-grained time series data. The framework integrates a set of techniques tailored for noisy and incomplete wearable data collected in real-world clinical studies: (1) singular spectrum analysis for extracting high-level features from daily features over the course of the study; (2) a set of daily features that are resilient to missing data in wearable time series data; (3) a K-Nearest Neighbors (KNN) method for imputing short missing heart rate segments; (4) the integration of patients' clinical characteristics and wearable features. We evaluated the feature engineering approach and machine learning models in a clinical study involving 61 patients undergoing pancreatic surgery. Linear support vector machine (SVM) with integrated feature engineering achieved an AUROC of 0.8802 for predicting post-operative readmission or severe complications, which significantly outperformed the existing rule-based model used in clinical practice and other state-of-the-art feature engineering approaches.

The time course of adaptation to distorted speech.

When confronted with unfamiliar or novel forms of speech, listeners' word recognition performance is known to improve with exposure, but data are lacking on the fine-grained time course of adaptation. The current study aims to fill this gap by investigating the time course of adaptation to several different types of distorted speech. Keyword scores as a function of sentence position in a block of 30 sentences were measured in response to eight forms of distorted speech. Listeners recognised twice as many words in the final sentence compared to the initial sentence with around half of the gain appearing in the first three sentences, followed by gradual gains over the rest of the block. Rapid adaptation was apparent for most of the eight distortion types tested with differences mainly in the gradual phase. Adaptation to sine-wave speech improved if listeners had heard other types of distortion prior to exposure, but no similar facilitation occurred for the other types of distortion. Rapid adaptation is unlikely to be due to procedural learning since listeners had been familiarised with the task and sentence format through exposure to undistorted speech. The mechanisms that underlie rapid adaptation are currently unclear.

Fine-grained time course of verb aspect processing.

Sentence processing is known to be highly incremental. Speakers make incremental commitments as the sentence unfolds, dynamically updating their representations based on the smallest pieces of information from the incoming speech stream. Less is known about linguistic processing on the sub-word level, especially with regard to abstract grammatical information. This study employs the Visual World Paradigm to investigate the processing of grammatical aspect by Russian-speaking adults (n = 124). Aspectual information is encoded relatively early within the Russian verb which makes this an ideal testing ground to investigate the incrementality of grammatical processing on the sub-word level. Participants showed preference for pictures of ongoing events when they heard sentences involving Imperfective verbs, and for pictures of completed events when they heard sentences involving Perfective verbs. Crucially, the analysis of the participants’ eye-movements showed that they exhibited preference for the target picture already before they heard the end of the verb. Moreover, the latency of this effect depended on where the aspectual information was encoded within the verb. These results indicate that the processing and integration of grammatical aspect information can happen rapidly and incrementally on a fine-grained word-internal level. Methodologically, the study draws together a set of analytical techniques which can be fruitfully applied to the analysis of effect latencies in a wide range of studies within the Visual World eye-tracking paradigm.

A machine learning pipeline for fuel-economical driving model

PurposeA cost-effective way to achieve fuel economy is to reinforce positive driving behaviour. Driving behaviour can be controlled if drivers can be alerted for behaviour that results in poor fuel economy. Fuel consumption must be tracked and monitored instantaneously rather than tracking average fuel economy for the entire trip duration. A single-step application of machine learning (ML) is not sufficient to model prediction of instantaneous fuel consumption and detection of anomalous fuel economy. The study designs an ML pipeline to track and monitor instantaneous fuel economy and detect anomalies.Design/methodology/approachThis research iteratively applies different variations of a two-step ML pipeline to the driving dataset for hatchback cars. The first step addresses the problem of accurate measurement and prediction of fuel economy using time series driving data, and the second step detects abnormal fuel economy in relation to contextual information. Long short-term memory autoencoder method learns and uses the most salient features of time series data to build a regression model. The contextual anomaly is detected by following two approaches, kernel quantile estimator and one-class support vector machine. The kernel quantile estimator sets dynamic threshold for detecting anomalous behaviour. Any error beyond a threshold is classified as an anomaly. The one-class support vector machine learns training error pattern and applies the model to test data for anomaly detection. The two-step ML pipeline is further modified by replacing long short term memory autoencoder with gated recurrent network autoencoder, and the performance of both models is compared. The speed recommendations and feedback are issued to the driver based on detected anomalies for controlling aggressive behaviour.FindingsA composite long short-term memory autoencoder was compared with gated recurrent unit autoencoder. Both models achieve prediction accuracy within a range of 98%–100% for prediction as a first step. Recall and accuracy metrics for anomaly detection using kernel quantile estimator remains within 98%–100%, whereas the one-class support vector machine approach performs within the range of 99.3%–100%.Research limitations/implicationsThe proposed approach does not consider socio-demographics or physiological information of drivers due to privacy concerns. However, it can be extended to correlate driver's physiological state such as fatigue, sleep and stress to correlate with driving behaviour and fuel economy. The anomaly detection approach here is limited to providing feedback to driver, it can be extended to give contextual feedback to the steering controller or throttle controller. In the future, a controller-based system can be associated with an anomaly detection approach to control the acceleration and braking action of the driver.Practical implicationsThe suggested approach is helpful in monitoring and reinforcing fuel-economical driving behaviour among fleet drivers as per different environmental contexts. It can also be used as a training tool for improving driving efficiency for new drivers. It keeps drivers engaged positively by issuing a relevant warning for significant contextual anomalies and avoids issuing a warning for minor operational errors.Originality/valueThis paper contributes to the existing literature by providing an ML pipeline approach to track and monitor instantaneous fuel economy rather than relying on average fuel economy values. The approach is further extended to detect contextual driving behaviour anomalies and optimises fuel economy. The main contributions for this approach are as follows: (1) a prediction model is applied to fine-grained time series driving data to predict instantaneous fuel consumption. (2) Anomalous fuel economy is detected by comparing prediction error against a threshold and analysing error patterns based on contextual information.

Quantification of ecosystem services supply-demand and the impact of demographic change on cultural services in Shenzhen, China

Previous studies on the supply and demand of ecosystem services (ES) mainly focused on inter-annual changes, and no studies have explored the impact of demographic change on the ES supply and demand on fine-grained time scales. Thus, taking Shenzhen as an example, the status of ES supply and demand, as well as diurnal population changes and their impacts on cultural services were analyzed at different time periods using mobile phone signaling data, ecological supply-demand ratio (ESDR), Geo-Informatic Tupu, InVEST model and buffer zone. The results showed that the population declines successively on workdays, weekends and holidays, and that the daytime population is greater than the nighttime. Water yield services can basically meet the demand in terms of quantity and spatial distribution, however, carbon sequestration and cultural services showed the opposite results. The main type of ESDR changes in cultural services are the mutual conversion of deficit and balance, and these are concentrated in areas with high forest coverage and small populations, but frequent population changes. In addition, when the fixed population is too large, the use of time-varying population data will conceal the impact of demographic changes on ES supply and demand, and other data are needed for auxiliary analysis. Overall, this study provides a new research perspective for the ES supply and demand and can provide a theoretical basis for refined sustainable urban management.