Ecological Divergence, Adaptive Diversification, and the Evolution of Social Signaling Traits: An Empirical Study in Arid Australian Lizards.

Humans convey their thoughts, emotions, and intentions through a concert of social displays: voice, facial expressions, hand gestures, and body posture, collectively referred to as social signals. Despite advances in machine perception, machines are unable to discern the subtleand momentary nuances that carry so much of the information and context of human communication.The encoding of conveyed information by social signals, particularly in nonverbal communication, is still poorly understood, and thus it is unclear how to teach machines touse such social signals to make them collaborative partners rather than tools that we use. A major obstacle to scientific progress in this direction is the inability to sense and measure the broad spectrum of behavioral cues in groups of interacting individuals, which hinders applying computational methods to model and understand social signals.In this thesis, we explore new approaches in sensing, measuring, and modeling social signals to ultimately endow machines with the ability to interpret nonverbal communication. This thesis starts by describing our exploration in building a massively multiviewsensor system, the Panoptic Studio, that can capture a broad spectrum of human social signaling—including voice, social formations, facial expressions, hand gestures, and body postures—among groups of multiple people. Second, leveraging this system equipped with more than 500 synchronized cameras, we then present a method to measure the subtle 3D movements of anatomical keypoints in face-to-face interaction, providing a new opportunityto computationally study social signals. In the last part of this thesis, we present a social signal prediction task to model nonverbal communication in a data-driven manner. We establish a new large-scale corpus from hundreds of participants containing various channels of social signal measurements. Leveraging this dataset, we verify that the social signals are predictive each other with strong correlations.

Monoaminergic activities of limbic regions are elevated during aggression: Influence of sympathetic social signaling

There is growing evidence of important variation in how animals age, in particular in how the expression of traits changes with age among different species and populations. However, less is known about variation within populations, which may include variation in ageing patterns between different types of individuals (e.g. sexes or distinct polymorphisms) and between different types of traits (e.g. general traits versus those used in social signalling contexts). We used 6years of longitudinal data to examine age-related changes in trait expression in a captive population of Gouldian finches (Erythrura gouldiae), a socially monogamous songbird with genetically determined colour morphs that differ in behaviour and physiology. We contrasted ageing patterns of different types of traits (social signalling vs. size-related) in both sexes and in two colour morphs, using a mixed model approach to account for both within- and between-individual effects. We found pronounced sex differences in how social signalling traits change with age, showing a quadratic pattern in males, but not changing with age in females. In contrast, we observed no sex-specific ageing patterns in size traits. We also found subtle morph differences in how size-related traits changed with age, with black morphs stable or increasing with age while red morphs showing a decline with age. Finally, we found an interesting sex by morph interaction in one important social signal (headband width). These results highlight the importance of using within-individual approaches to understand ageing patterns across types of individuals (sex, morph, etc.) and the need for further research on the ageing patterns of traits that may experience different selective pressures.

ABSTRACTThe dissemination of information is a basic element of group cohesion. In honey bees (Apis mellifera Linnaeus 1758), like in other social insects, the principal method for colony-wide information exchange is communication via pheromones. This medium of communication allows multiple individuals to conduct tasks critical to colony survival. Social signaling also establishes conflict at the level of the individual who must trade-off between attending to the immediate environment or the social demand. In this study we examined this conflict by challenging highly social worker honey bees, and less social male drone honey bees undergoing aversive training by presenting them with a social stress signal (isopentyl acetate, IPA). We utilized IPA exposure methods that caused lower learning performance in appetitive learning in workers. Exposure to isopentyl acetate (IPA) did not affect performance of drones and had a dose-specific effect on worker response, with positive effects diminishing at higher IPA doses. The IPA effects are specific because non-social cues, such as the odor cineole, improve learning performance in drones, and social homing signals (geraniol) did not have a discernible effect on drone or worker performance. We conclude that social signals do generate conflict and that response to them is dependent on signal relevance to the individual as well as the context. We discuss the effect of social signal on learning both related to its social role and potential evolutionary history.

Social Information Signaling by Neurons in Primate Striatum

As complex beings, humans communicate in complex ways, relying on a range of faculties to encode and decode social messages. Some aptitudes are innate, based on one's biological characteristics, whereas others are acquired, varying according to one's social and cultural experiences. As we explain in this chapter, each of us uses a combination of biological and sociocultural processes to produce and interpret social signals. Our goal is to introduce some of the forms that these processes can take. We begin this chapter with an overview of social signals and a comparison between the biological and sociocultural processes underlying their production and interpretation. Next, we explore three examples of biologically processed social signals, and then examine sociocultural processing of the same signals. We conclude the chapter by discussing some ways in which biological and sociocultural processes interact. The Nature of Social Signals Communicators depend on a wide variety of social signals to make sense of the world around them. Poggi and D'Errico (2011) define a signal as “any perceivable stimulus from which a system can draw some meaning” and a social signal as “a communicative or informative signal which, either directly or indirectly, provides information about ‘social facts,’ that is, about social interactions, social attitudes, social relations and social emotions” (Poggi & D'Errico, 2011: 189). Social interactions are situations in which people perform reciprocal social actions, such as a game, a surgical procedure, an orchestral performance, or a conflict. Social attitudes are people's tendencies to behave in a particular way toward another person or group and include elements such as beliefs, opinions, evaluations, and emotions. Social relations are relationships of interdependent goals between two or more people. Finally, social emotions include those emotions that (1) we feel toward someone else, such as admiration and envy; (2) are easily transmitted from one person to another, such as enthusiasm and panic; and/or (3) are self-conscious, such as pride and shame. As noted, humans use both biological and sociocultural processes to produce and interpret social signals.

Social Cognition in Humans

Faced with the cutbacks in public funding and the changes taking place in the governance and funding models in the cultural sector, museums must rise to the challenge of devising and implementing strategies to obtain resources from a range of sources and thus reduce public sector dependence. Based on a sample of museums from various countries which use private funding, the present work examines different signals that can impact on private fundraising from donors and sponsors: social signals (reputation and social performance) and financial signals (accountability and fundable projects). The results reveal that whereas donors are concerned with all kinds of social and financial signals, sponsors are mainly attracted by reputation and fundable projects. The study also draws a distinction between small and large museums. While the former should offer private funders flexibility in funding, the latter need to evidence social achievements as well as financial features to attract funders.

Despite the recent increase in second-person neuroscience research, it is still hard to understand which neurocognitive mechanisms underlie real-time social behaviours. Here, we propose that social signalling can help us understand social interactions both at the single- and two-brain level in terms of social signal exchanges between senders and receivers. First, we show how subtle manipulations of being watched provide an important tool to dissect meaningful social signals. We then focus on how social signalling can help us build testable hypotheses for second-person neuroscience with the example of imitation and gaze behaviour. Finally, we suggest that linking neural activity to specific social signals will be key to fully understand the neurocognitive systems engaged during face-to-face interactions.

How anthropogenic change affects animal social behavior, including communication is an important question. Urban noise often drives shifts in acoustic properties of signals but the consequences of noise for the honesty of signals—that is, how well they predict signaler behavior—is unclear. Here we examine whether honesty of aggressive signaling is compromised in male urban song sparrows (Melospiza melodia). Song sparrows have two honest close-range signals: the low amplitude soft songs (an acoustic signal) and wing waves (a visual signal), but whether the honesty of these signals is affected by urbanization has not been examined. If soft songs are less effective in urban noise, we predict that they should predict attacks less reliably in urban habitats compared to rural habitats. We confirmed earlier findings that urban birds were more aggressive than rural birds and found that acoustic noise was higher in urban habitats. Urban birds still sang more soft songs than rural birds. High rates of soft songs and low rates of loud songs predicted attacks in both habitats. Thus, while urbanization has a significant effect on aggressive behaviors, it might have a limited effect on the overall honesty of aggressive signals in song sparrows. We also found evidence for a multimodal shift: urban birds tended to give proportionally more wing waves than soft songs than rural birds, although whether that shift is due to noise-dependent plasticity is unclear. These findings encourage further experimental study of the specific variables that are responsible for behavioral change due to urbanization.Soft song, the low amplitude songs given in close range interactions, is an honest threat signal in urban song sparrows. Given its low amplitude, soft songs may be a less effective signal in noisy urban habitats. However, we found that soft song remained an honest signal predicting attack in urban habitats. We also found that birds may use more visual signals (rapid fluttering of wings) in urban habitats to avoid masking from acoustic noise.

In this chapter we focus on systematization, analysis, and discussion of recent trends in machine learning methods for Social signal processing (SSP) (Pentland, 2007). Because social signaling is often of central importance to subconscious decision making that affects everyday tasks (e.g., decisions about risks and rewards, resource utilization, or interpersonal relationships), the need for automated understanding of social signals by computers is a task of paramount importance. Machine learning has played a prominent role in the advancement of SSP over the past decade. This is, in part, due to the exponential increase of data availability that served as a catalyst for the adoption of a new data-driven direction in affective computing. With the difficulty of exact modeling of latent and complex physical processes that underpin social signals, the data has long emerged as the means to circumvent or supplement expert- or physics-based models, such as the deformable musculoskeletal models of the human body, face, or hands and its movement, neuro-dynamical models of cognitive perception, or the models of the human vocal production. This trend parallels the role and success of machine learning in related areas, such as computer vision (c.f., Poppe, 2010; Wright et al., 2010; Grauman & Leibe, 2011) or audio, speech and language processing (c.f., Deng & Li, 2013), that serve as the core tools for analytic SSP tasks. Rather than emphasize the exhaustive coverage of the many approaches to data-driven SSP, which can be found in excellent surveys (Vinciarelli, Pantic, & Bourlard, 2009; Vinciarelli et al., 2012), we seek to present the methods in the context of current modeling challenges. In particular, we identify and discuss two major modeling directions:

Face-to-face communication conveys social context as well as words. It is this social signaling that allows new information to be smoothly integrated into a shared, group-wide understanding. Social signaling includes signals of interest, determination, friendliness, boredom, and other "attitudes" toward a social situation. Psychologists speculate that social signaling may have evolved as a way to establish hierarchy and group cohesion because social signaling functions as a subconscious discussion about relationships, resources, risks, and rewards. In many situations the nonlinguistic signals that serve as the basis for this social discussion are just as important as conscious content for determining human behavior. In what follows we discuss challenges in exploratory processing of social signals and tools that allow us to predict human behavior and sometimes exceed even expert human capabilities. These tools potentially permit computer and communications systems to support social and organizational roles instead of viewing the individual as an isolated entity. Example applications include automatically patching people into socially important conversations, instigating conversations among people in order to build a more solid social network, and reinforcing family ties.

The behavioral neuroscience of anuran social signal processing

Across multiple sectors training programmes aim to help learners improve their communication skills. It is well recognised that non-verbal ‘social signals’ play an important role in communication effectiveness. Previous research in the social signalling domain meticulously observed hours of videos and conducted observational studies to identify these social signals. This resulted in subjective inferences about human emotions. The aim of the current research is to investigate whether social signals can be detected and trained using automated technology in a person-to-person training context in three stages; exploratory stage, feedback design stage and an experimental stage. This research will allow trainers to provide objective feedback to trainees about their performance with a clear criterion. This research will also explore the best way to feedback signals detected and whether this is effective and actionable. Further long-term benefits of this research might contribute to the eventual development of a training avatar.

In Focus: Formica, V., Donald, H., Marti, H., Irgebay, Z., Brodie III, E. Social network position experiences more variable selection than weaponry in wild subpopulations of forked fungus beetles. Journal of Animal Ecology, 90, 168-182, https://doi.org/10.1111/1365-2656.13322. That social network traits can exhibit consistent-individual differences among individuals and confer a fitness benefit or cost is increasingly well-established. However, how selection-natural or sexual-affects those social traits and at what scale remains an open question. In this Special Feature, Formica and colleagues employ a meta-population of forked fungus beetles to test and contrast whether sexual selection on social network traits contrasted to morphological traits occurs at the local (soft) or global (hard) scales. The authors demonstrate that morphological traits are largely under hard directional positive selection, whereas social traits are under soft and variable selection. The findings are compelling and raise interesting discussion of multi-level selection and the evolution of social traits in a meta-population.