Action Parsing Research Articles

A Computational Complexity Perspective on Segmentation as a Cognitive Subcomputation.

Computational feasibility is a widespread concern that guides the framing and modeling of natural and artificial intelligence. The specification of cognitive system capacities is often shaped by unexamined intuitive assumptions about the search space and complexity of a subcomputation. However, a mistaken intuition might make such initial conceptualizations misleading for what empirical questions appear relevant later on. We undertake here computational-level modeling and complexity analyses of segmentation - a widely hypothesized subcomputation that plays a requisite role in explanations of capacities across domains, such as speech recognition, music cognition, active sensing, event memory, action parsing, and statistical learning - as a case study to show how crucial it is to formally assess these assumptions. We mathematically prove two sets of results regarding computational hardness and search space size that may run counter to intuition, and position their implications with respect to existing views on thesubcapacity.

Tradition and invention: The bifocal stance theory of cultural evolution.

Cultural evolution depends on both innovation (the creation of new cultural variants by accident or design) and high-fidelity transmission (which preserves our accumulated knowledge and allows the storage of normative conventions). What is required is an overarching theory encompassing both dimensions, specifying the psychological motivations and mechanisms involved. The bifocal stance theory (BST) of cultural evolution proposes that the co-existence of innovative change and stable tradition results from our ability to adopt different motivational stances flexibly during social learning and transmission. We argue that the ways in which instrumental and ritual stances are adopted in cultural transmission influence the nature and degree of copying fidelity and thus also patterns of cultural spread and stability at a population level over time. BST creates a unifying framework for interpreting the findings of otherwise seemingly disparate areas of enquiry, including social learning, cumulative culture, overimitation, and ritual performance. We discuss the implications of BST for competing by-product accounts which assume that faithful copying is merely a side-effect of instrumental learning and action parsing. We also set out a novel "cultural action framework" bringing to light aspects of social learning that have been relatively neglected by behavioural ecologists and evolutionary psychologists and establishing a roadmap for future research on this topic. The BST framework sheds new light on the cognitive underpinnings of cumulative cultural change, selection, and spread within an encompassing evolutionary framework.

Action Parsing-Driven Video Summarization Based on Reinforcement Learning

How to manage, store, and index large numbers of videos is an urgent problem to be solved. Although there are many video summarization models achieving good results, models based on low-level features cannot summarize important semantic information and models based on semantic analysis need related text descriptions that do not exist for most videos. As a consequence, the mining semantic information contained in the video itself is a more feasible way. In this paper, we propose an action parsing-driven video summarization model based on reinforcement learning. The model is mainly divided into two parts, video cut by action parsing and video summarization based on reinforcement learning. In the first part, a sequential multiple instance learning model is trained with weakly annotated data to solve the problem of full annotation’s time consuming and weak annotation’s ambiguity. In the second part, we design a deep recurrent neural network-based video summarization model that selects the most distinguishable frames comparing with other actions. Meanwhile, the quality of the extracted key frames could be evaluated by the categorization accuracy. Experiments and comparison with state-of-the-art methods demonstrate the advantage of the proposed approach.

Deep Action Parsing in Videos With Large-Scale Synthesized Data.

Action parsing in videos with complex scenes is an interesting but challenging task in computer vision. In this paper, we propose a generic 3D convolutional neural network in a multi-task learning manner for effective Deep Action Parsing (DAP3D-Net) in videos. Particularly, in the training phase, action localization, classification, and attributes learning can be jointly optimized on our appearance-motion data via DAP3D-Net. For an upcoming test video, we can describe each individual action in the video simultaneously as: Where the action occurs, What the action is, and How the action is performed. To well demonstrate the effectiveness of the proposed DAP3D-Net, we also contribute a new Numerous-category Aligned Synthetic Action data set, i.e., NASA, which consists of 200 000 action clips of over 300 categories and with 33 pre-defined action attributes in two hierarchical levels (i.e., low-level attributes of basic body part movements and high-level attributes related to action motion). We learn DAP3D-Net using the NASA data set and then evaluate it on our collected Human Action Understanding data set and the public THUMOS data set. Experimental results show that our approach can accurately localize, categorize, and describe multiple actions in realistic videos.

Performance error-related activity in monkey striatum during social interactions.

Monitoring our performance is fundamental to motor control while monitoring other’s performance is fundamental to social coordination. The striatum is hypothesized to play a role in action selection, action initiation, and action parsing, but we know little of its role in performance monitoring. Furthermore, the striatum contains neurons that respond to own and other’s actions. Therefore, we asked if striatal neurons signal own and conspecific’s performance errors. Two macaque monkeys sitting across a touch-sensitive table in plain view of each other took turns performing a simple motor task to obtain juice rewards while we recorded single striatal neurons from one monkey at a time. Both monkeys made more errors after individually making an error but made fewer errors after a conspecific error. Thus, monkeys’ behavior was influenced by their own and their conspecific’s past behavior. A population of striatal neurons responded to own and conspecific’s performance errors independently of a negative reward prediction error signal. Overall, these data suggest that monkeys are influenced by social errors and that striatal neurons signal performance errors. These signals might be important for social coordination, observational learning and adjusting to an ever-changing social landscape.

Rational over-imitation: Preschoolers consider material costs and copy causally irrelevant actions selectively

Children’s strong tendency to over-imitate – i.e., to reproduce causally irrelevant actions – presents a well-documented, yet puzzling, phenomenon. On first sight this instrumentally inefficient behavior seems maladaptive and different accounts have been put forward to explain it. Causal accounts claim that children are misled by an adult’s demonstration, mistake the superfluous actions as causally necessary, and therefore imitate them. Other accounts emphasize cognitive-motivational aspects underlying over-imitation, e.g. social motivations to affiliate with the model, or to adhere to normative conventions. Since all accounts predict the occurrence of over-imitation under typical conditions, different parameters and circumstances have to be considered to distinguish between them. Thus, we investigated children’s over-imitation and their spontaneous verbal reactions to a puppet’s behavior, in contexts in which a causally irrelevant action either led to the destruction of a valuable object belonging to the experimenter, or not. In addition, children saw the full action sequence being demonstrated either with an instrumental or a conventional focus. Causal accounts predict no flexibility across these contexts, because over-imitation is said to occur automatically. Normative accounts claim that different normative considerations affect children’s behavior and action parsing, and therefore predict different response patterns across conditions. We found that over-imitation was less frequent in costly and instrumental conditions. Children criticized the puppet for omitting irrelevant actions more often in the non-costly condition, but criticized her more often for performing irrelevant actions in the costly condition, often expressing their moral concern. The results support the rational normative action interpretation account of over-imitation.

Parsing of action sequences: A neural dynamics approach

AbstractParsing of action sequences is the process of segmenting observed behavior into individual actions. In robotics, this process is critical for imitation learning from observation and for representing an observed behavior in a form that may be communicated to a human. In this paper, we develop a model for action parsing, based on our understanding of principles of grounded cognitive processes, such as perceptual decision making, behavioral organization, and memory formation.We present a neural-dynamic architecture, in which action sequences are parsed using a mathematical and conceptual framework for embodied cognition—the Dynamic Field Theory. In this framework, we introduce a novel mechanism, which allows us to detect and memorize actions that are extended in time and are parametrized by the target object of an action. The core properties of the architecture are demonstrated in a set of simple, proof-of-concept experiments.

Problems for the cognitive-depletion model of religious interactions

Schoejdt, Sorensen, Nielbo, Xygalatas, Mitkidis and Bulbulia put forward a new cognitive model of religious interactions, which we depict in Figure one. The model stresses three common features of such interactions: the requirement to suppress emotion expression, the exposure to ritualized behavior and the presence of a charismatic authority (1). These features have different cognitive effects involving different sorts of cognitive-resource depletion (2). The regulation of emotion expression drives one’s attention to inward phenomena, compromising one’s ability to form episodic memories of the external, observable aspects of rituals. Ritualized behavior precludes the ordinary, means-end analysis of action and demands low-level action parsing. These two effects impede the interpretation of ritualized behavior or other observable aspects of rituals (3), which makes one susceptible to authoritative interpretive discourses (4), which in turn promote collective religious-symbolic interpretations (5). The presence of a charismatic authority has a third cognitive effect involving cognitive-resource depletion—lack of error monitoring concerning the evidence disconfirming an authoritative interpretation. Since a charismatic authority is an authority, it can play two roles: a conjoint one, by adding another force for the establishment of collective religious-symbolic interpretations (4-1→2→5); an independent one, by persuading individuals to align their interpretations of a ritual with the authoritative one (1→2→5). Thus, via cognitive-resource depletion, the three features contribute to the same effect, which constitutes their social function—to

Why ritualized behavior? Precaution Systems and action parsing in developmental, pathological and cultural rituals

Ritualized behavior, intuitively recognizable by its stereotypy, rigidity, repetition, and apparent lack of rational motivation, is found in a variety of life conditions, customs, and everyday practices: in cultural rituals, whether religious or non-religious; in many children's complicated routines; in the pathology of obsessive-compulsive disorders (OCD); in normal adults around certain stages of the life-cycle, birthing in particular. Combining evidence from evolutionary anthropology, neuropsychology and neuroimaging, we propose an explanation of ritualized behavior in terms of an evolved Precaution System geared to the detection of and reaction to inferred threats to fitness. This system, distinct from fear-systems geared to respond to manifest danger, includes a repertoire of clues for potential danger as well as a repertoire of species-typical precautions. In OCD pathology, this system does not supply a negative feedback to the appraisal of potential threats, resulting in doubts about the proper performance of precautions, and repetition of action. Also, anxiety levels focus the attention on low-level gestural units of behavior rather than on the goal-related higher-level units normally used in parsing the action-flow. Normally automatized actions are submitted to cognitive control. This "swamps" working memory, an effect of which is a temporary relief from intrusions but also their long-term strengthening. Normal activation of this Precaution System explains intrusions and ritual behaviors in normal adults. Gradual calibration of the system occurs through childhood rituals. Cultural mimicry of this system's normal input makes cultural rituals attention-grabbing and compelling. A number of empirical predictions follow from this synthetic model.

Ritualized behavior in animals and humans: Time, space, and attention

A study of the organization of obsessive-compulsive disorder (OCD) rituals in time and space illuminates a postulated mechanism on shifting focus in action parsing, from mid-ranged actions to finer movements (gestures). Performance of OCD rituals also involves high concentration rather than the automated, less attended performance of rituals in normal and stereotyped behaviors in animals and humans.