Non-sequential Models Research Articles

Evaluation of deep learning-based depression detection using medical claims data

Human accuracy in diagnosing psychiatric disorders is still low. Even though digitizing health care leads to more and more data, the successful adoption of AI-based digital decision support (DDSS) is rare. One reason is that AI algorithms are often not evaluated based on large, real-world data. This research shows the potential of using deep learning on the medical claims data of 812,853 people between 2018 and 2022, with 26,973,943 ICD-10-coded diseases, to predict depression (F32 and F33 ICD-10 codes). The dataset used represents almost the entire adult population of Estonia. Based on these data, to show the critical importance of the underlying temporal properties of the data for the detection of depression, we evaluate the performance of non-sequential models (LR, FNN), sequential models (LSTM, CNN-LSTM) and the sequential model with a decay factor (GRU-Δt, GRU-decay). Furthermore, since explainability is necessary for the medical domain, we combine a self-attention model with the GRU decay and evaluate its performance. We named this combination Att-GRU-decay. After extensive empirical experimentation, our model (Att-GRU-decay), with an AUC score of 0.990, an AUPRC score of 0.974, a specificity of 0.999 and a sensitivity of 0.944, proved to be the most accurate. The results of our novel Att-GRU-decay model outperform the current state of the art, demonstrating the potential usefulness of deep learning algorithms for DDSS development. We further expand this by describing a possible application scenario of the proposed algorithm for depression screening in a general practitioner (GP) setting—not only to decrease healthcare costs, but also to improve the quality of care and ultimately decrease people’s suffering.

A hierarchical residual network with compact triplet-center loss for sketch recognition

With the widespread use of touch-screen devices, it is more and more convenient for people to draw sketches on screen. This results in the demand for automatically understanding the sketches. Thus, the sketch recognition task becomes more significant than before. To accomplish this task, it is necessary to solve the critical issue of improving the distinction of the sketch features. To this end, we have made efforts in three aspects. First, a novel multi-scale residual block is designed. Compared with the conventional basic residual block, it can better perceive multi-scale information and reduce the number of parameters during training. Second, a hierarchical residual structure is built by stacking multi-scale residual blocks in a specific way. In contrast with the single-level residual structure, the learned features from this structure are more sufficient. Last but not least, the compact triplet-center loss is proposed specifically for the sketch recognition task. It can solve the problem that the triplet-center loss does not fully consider too large intra-class space and too small inter-class space in sketch field. By studying the above modules, a hierarchical residual network as a whole is proposed for sketch recognition and evaluated on Tu-Berlin benchmark thoroughly. The experimental results show that the proposed network outperforms most of baseline methods and it is excellent among non-sequential models at present.

Can subjective pain be inferred from objective physiological data? Evidence from patients with sickle cell disease.

Patients with sickle cell disease (SCD) experience lifelong struggles with both chronic and acute pain, often requiring medical interventMaion. Pain can be managed with medications, but dosages must balance the goal of pain mitigation against the risks of tolerance, addiction and other adverse effects. Setting appropriate dosages requires knowledge of a patient’s subjective pain, but collecting pain reports from patients can be difficult for clinicians and disruptive for patients, and is only possible when patients are awake and communicative. Here we investigate methods for estimating SCD patients’ pain levels indirectly using vital signs that are routinely collected and documented in medical records. Using machine learning, we develop both sequential and non-sequential probabilistic models that can be used to infer pain levels or changes in pain from sequences of these physiological measures. We demonstrate that these models outperform null models and that objective physiological data can be used to inform estimates for subjective pain.

The effect of jumping modes on various automata models

Recently, new types of non-sequential machine models have been introduced and studied, such as jumping automata and one-way jumping automata. We study the abilities and limitations of (finite, pushdown and linear bounded) automata with these 2 jumping modes of tape heads with respect to how they affect the class of accepted languages. We provide adapted versions of pumping lemmas and other methods to determine whether a language is accepted by a machine with jumping mode. Using these methods we establish the inclusion or incomparability relationships among the classes of languages defined by the new machines and their classical counterparts. We also study the closure properties of the resulting language classes and show that under most fundamental language operations, these classes are not closed.

IR2V EC

We propose IR2V EC , a Concise and Scalable encoding infrastructure to represent programs as a distributed embedding in continuous space. This distributed embedding is obtained by combining representation learning methods with flow information to capture the syntax as well as the semantics of the input programs. As our infrastructure is based on the Intermediate Representation (IR) of the source code, obtained embeddings are both language and machine independent. The entities of the IR are modeled as relationships, and their representations are learned to form a seed embedding vocabulary . Using this infrastructure, we propose two incremental encodings: Symbolic and Flow-Aware . Symbolic encodings are obtained from the seed embedding vocabulary , and Flow-Aware encodings are obtained by augmenting the Symbolic encodings with the flow information. We show the effectiveness of our methodology on two optimization tasks (Heterogeneous device mapping and Thread coarsening). Our way of representing the programs enables us to use non-sequential models resulting in orders of magnitude of faster training time. Both the encodings generated by IR2V EC outperform the existing methods in both the tasks, even while using simple machine learning models. In particular, our results improve or match the state-of-the-art speedup in 11/14 benchmark-suites in the device mapping task across two platforms and 53/68 benchmarks in the thread coarsening task across four different platforms. When compared to the other methods, our embeddings are more scalable , is non-data-hungry , and has better Out-Of-Vocabulary (OOV) characteristics .

E-mail classification with machine learning and word embeddings for improved customer support

Classifying e-mails into distinct labels can have a great impact on customer support. By using machine learning to label e-mails, the system can set up queues containing e-mails of a specific category. This enables support personnel to handle request quicker and more easily by selecting a queue that match their expertise. This study aims to improve a manually defined rule-based algorithm, currently implemented at a large telecom company, by using machine learning. The proposed model should have higher F_1-score and classification rate. Integrating or migrating from a manually defined rule-based model to a machine learning model should also reduce the administrative and maintenance work. It should also make the model more flexible. By using the frameworks, TensorFlow, Scikit-learn and Gensim, the authors conduct a number of experiments to test the performance of several common machine learning algorithms, text-representations, word embeddings to investigate how they work together. A long short-term memory network showed best classification performance with an F_1-score of 0.91. The authors conclude that long short-term memory networks outperform other non-sequential models such as support vector machines and AdaBoost when predicting labels for e-mails. Further, the study also presents a Web-based interface that were implemented around the LSTM network, which can classify e-mails into 33 different labels.

On-line human activity recognition from audio and home automation sensors: Comparison of sequential and non-sequential models in realistic Smart Homes1

Automatic human Activity Recognition (AR) is an important process for the provision of context-aware services in smart spaces such as voice-controlled smart homes. This paper presents an on-line Activities of Daily Living (ADL) recognition method for automatic identification within homes in which m ultiple sensors, actuators and automation equipment coexist, including audio sensors. Three sequence-based models are presented and compared: a Hidden Markov Model (HMM), Conditional Random Fields (CRF) and a sequential Markov Logic Network (MLN). These methods have been tested in two real Smart Homes thanks to experiments involving more than 30 participants. Their results were compared to those of three non-sequential models: a Support Vector Machine (SVM), a Random Forest (RF) and a non-sequential MLN. This comparative study shows that CRF gave the best results for on-line activity recognition from non-visual, audio and home automation sensors.

Re-Thinking Group Development in Adventure Programming: A Qualitative Examination

The purpose of the study was to re-examine the efficacy of Tuckman’s (1965) model of group development in adventure programming. More specifically, the researchers were interested in comparing Tuckman’s traditional sequential model to more contemporary non-sequential and integrative models that have emerged as alternative ways to conceptualize the process of group development. The researchers used a case study approach (Merriam, 2001) and techniques characteristic of naturalistic inquiry (Lincoln & Guba, 1985) to explore the nature of group development among participants in a weeklong sea kayaking expedition. The researchers concluded that more contemporary models of group development more accurately explained the process of group development in this case study than Tuckman’s (1965) traditional model of group development.

Pseudo Amino Acid Composition and its Applications in Bioinformatics, Proteomics and System Biology

With the avalanche of protein sequences generated in the post-genomic age, it is highly desired to develop automated methods for efficiently identifying various attributes of uncharacterized proteins. This is one of the most important tasks facing us today in bioinformatics, and the information thus obtained will have important impacts on the development of proteomics and system biology. To realize that, one of the keys is to find an effective model to represent the sample of a protein. The most straightforward model in this regard is its entire amino acid sequence; however, the entire sequence model would fail to work when the query protein did not have significant homology to proteins of known characteristics. Thus, various non-sequential models or discrete models were proposed. The simplest discrete model is the amino acid (AA) composition. Using it to represent a protein, however, all the sequence-order information would be completely lost. To cope with such a dilemma, the concept of pseudo amino acid (PseAA) composition was introduced. Its essence is to keep using a discrete model to represent a protein yet without completely losing its sequence-order information. Therefore, in a broad sense, the PseAA composition of a protein is actually a set of discrete numbers that is derived from its amino acid sequence and that is different from the classical AA composition and able to harbour some sort of sequence order or pattern information. Ever since the first PseAA composition was formulated to predict protein subcellular localization and membrane protein types, it has stimulated many different modes of PseAA composition for studying various kinds of problems in proteins and proteins-related systems. In this review, we shall give a brief and systematic introduction of various modes of PseAA composition and their applications. Meanwhile, the challenges for finding the optimal PseAA composition are also briefly discussed.

Algorithm of mining sequential patterns for web personalization services

This paper focuses on the requirements of web personalization service for sequential patterns and sequential mining algorithms. Previous sequential mining algorithms treated sequential patterns uniformly, but individual patterns in sequences often have different importance weights. To solve this problem, we propose a new algorithm to identify weighted maximal frequent sequential patterns. First, frequency of frequent single items is used to calculate the weights of frequent sequences. Then, the frequent weighted sequence is defined, leading not only to the discovery of important maximal sequences, but the property of anti-monotony. Web usage mining has been used effectively to inform web personalization and recommender systems, and this new algorithm provides an effective method for optimizing these services. A variety of recommendation frameworks have been proposed previously, including some based on non-sequential models such as association rules, as well as sequential models. In this paper, we present a hybrid web personalization system based on clustering and contiguous sequential patterns. Our system clusters log files to determine the basic architecture of websites, and for each cluster, we use contiguous sequential pattern mining to further optimize the topologies of websites. Finally, we propose two evaluating parameters to test the performance of our system.

From contexts to constructs: the use of grounded theory in operationalising contingent process models

In software and systems engineering, process models have been suggested that partly develop artifacts and then deploy them to evaluate the degree of their success in satisfying requirements, using the results of the evaluation to inform further engineering efforts. This paper discusses the important role of evaluation in non-sequential process models, and suggests an interpretive framework for effecting such evaluation, as well as the determination of risks and constraints to operationalise such models. We outline a particular mode of the Grounded Systems Engineering Methodology (GSEM) which aims at supporting the application of non-sequential process models, thus enabling them to be more responsive to the development context. GSEM utilises the philosophy and procedures of the Grounded Theory method, an interpretive research strategy which is being widely used for the inductive analysis of qualitative data in theory building. An illustrative case study is used to show how the framework can be applied in practice.

Group development (I): A review and synthesis of development models.

This is the first of two parts that examine the issue of group development and its impact on the study design of group support systems (GSS). We review the various models of group development, analyze the sources of differences among these models, and synthesize common themes across various models. The paper concludes with a meta-framework for understanding group development; this framework highlights the two areas of focus that have dominated group development research in the past: group processes and outcomes. The second paper will build on the ideas developed here and discuss the implications of group development for GSS research. Previous research on group behavior suggests that groups change over time; patterns of change, referred to as group development models, have been an important area of study for the past four decades. For the first three of these decades, unitary models of group development were very popular; that is, the notion that all groups go through a certain series of predefined stages. In the last decade, however, researchers have cast doubt on such unitary models of group development. Nonsequential models that recognize the uniqueness of each group (and consequently reject the idea of a single, predetermined series of stages) have become increasingly popular. This paper examines the implications of these issues for researchers and managers of groups. It also attempts to serve as the foundation for the propositions developed in the next paper, in which the relevance of group development for GSS research and practice are discussed.

Selection of model structure for road accident severity

The presence of rare events in the accident data and transferability of error from one severity level to the other are just two of the issues in modelling accident severity. This paper explores the implications of different model structures and evaluation criteria using a common data set. Evaluation criteria are statistical fit of submodels, stability and validity of risk factors, overall model prediction, and case-specific model predictions. Model structures considered were the sequence of injury severity in the model structure, sequential versus nonsequential models, degree of aggregation, and case control approach. The study concluded that all the different models produce similar results. There is no indication of transferability of error between submodels in the sequential model. It was concluded that a disaggregate model is preferred to an aggregate model given the similarity of results. Key words: road accident severity, personal injury, logit, model choice.

The Implications of Group Development and History for Group Support System Theory and Practice

The psychology, sociology, and management literature is rich with theory on the topic of group development. Three general categories of developmental models have been proposed: progressive models, cyclical models, and nonsequential models. These models predict that group performance, member behavior, and task focus change as the group matures and develops. Groups are integral components of organizations. Group support systems (GSSs) have been created to assist groups in performing their tasks and activities. Numerous research studies have been conducted to identify key variables that influence group performance in GSS environments. Many of these studies have produced mixed and, in some cases, conflicting results. This may be due partially to a lack of attention to the issue of group development. Few studies have looked at the influence that group development and group history have on group behavior and performance. Only two GSS studies have been conducted on the effects of group development and group history on group process, performance, and efficiency. To facilitate additional research into this area, a research framework that incorporates a mechanism for representing this development process is proposed In addition, a number of propositions implied by the relationships represented in this framework are presented and discussed

D-Continuity and Petri’s Axioms of Concurrency for Nonsequential Process Models

A non-sequential process can be modelled by a partially ordered set. Conversely, one is led to study the properties to be fulfilled by a poset so that it can reasonably be viewed as the model of a non-sequential process. To this end, C.A. Petri has proposed a set of con currency axioms and a related property called D-continuity, a generalised version for partially ordered sets of Dedekind’s completeness property of the real numbers. In this paper we study Petri’s axioms of concurrency and some of their interdependencies. We also derive several characterisations of D-continuity and exhibit its relation with the axioms of concurrency. Furthermore we apply our work to Petri nets: we introduce occurrence nets, some special posets which model the processes of a system net and we present their relations to D-continuity and the axioms of con currency. Finally we identify the class of the system nets whose processes are D-continuous and satisfy the axioms of concurrency.