Human Interactive Communication Research Articles

Special issue on robot and human interactive communication

Special issue on robot and human interactive communication

An adhesive, highly stretchable and low-hysteresis alginate-based conductive hydrogel strain sensing system for motion capture

A strain sensor stands as an indispensable tool for capturing intricate motions in various applications, ranging from human motion monitoring to electronic skin and soft robotics. However, existing strain sensors still face difficulties in simultaneously achieving superior sensing performance sufficing for practical applications like high stretchability and low hysteresis, as well as seamless device fabrication like desirable interfacial adhesion and system-level integration. Herein, we develop a highly stretchable and low-hysteresis strain sensor with adhesive poly(3,4-ethylenedioxythiophene):poly(styrene sulfonate) (PEDOT:PSS)/polyacrylamide (PAAm)–sodium alginate (SA) composite hydrogel, allowing the successful construction of a wireless motion capture sensing system that can provide precise data collection within a large deformation range. The resultant composite hydrogel displays favorable interfacial adhesion and robust mechanical stability, and the fabricated strain sensor demonstrates a wide working strain range (up to 500%) with high sensitivity (gauge factor = 11) and ultra-low hysteresis (1.52%), outperforming previous PEDOT-based hydrogel strain sensors. Enabled by the intriguing material properties and high sensing performance, we further demonstrate the fabrication and integration of a wireless motion capture sensing system for diverse applications like human motion monitoring, gesture recognition, and interactive communication.

Open-set 3D semantic instance maps for vision language navigation – O3D-SIM

Humans excel at forming mental maps of their surroundings, equipping them to understand object relationships and navigate based on language queries. Our previous work SI Maps (Nanwani L, Agarwal A, Jain K, et al. Instance-level semantic maps for vision language navigation. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE; 2023 Aug.) showed that having instance-level information and the semantic understanding of an environment helps significantly improve performance for language-guided tasks. We extend this instance-level approach to 3D while increasing the pipeline's robustness and improving quantitative and qualitative results. Our method leverages foundational models for object recognition, image segmentation, and feature extraction. We propose a representation that results in a 3D point cloud map with instance-level embeddings, which bring in the semantic understanding that natural language commands can query. Quantitatively, the work improves upon the success rate of language-guided tasks. At the same time, we qualitatively observe the ability to identify instances more clearly and leverage the foundational models and language and image-aligned embeddings to identify objects that, otherwise, a closed-set approach wouldn't be able to identify. Project Page - https://smart-wheelchair-rrc.github.io/o3d-sim-webpage

A Framework for Guiding the Design of Interactive Systems for Communication through Activities in Higher Learning Institutions

Human-computer interaction is paramount in the process of designing interactive-computer systems with high capabilities of guaranteeing users’ satisfaction. However, lack of suitable frameworks complicates their effective design. This study investigated suitable frameworks capable of guiding appropriate design of interactive systems for communication through activities (ISCA) in higher learning institutions in Tanzania. These systems are necessary for augmenting communication thereby overcoming persistent challenges related to their reliance on face-to-face, phone-based, and social networking websites communications. This study used a thorough user centred design approach where a framework for human activity design-centred (HADC) was employed. Authors qualitatively analysed data on communication and interaction issues from participants involved. Design science research was then combined with activity theory to develop a four-phased methodology which was used to design an intended human activity interactive communication (HAIC) framework. The findings confirmed that interactions design techniques based on human-activities undertaken are fundamental in designing communication frameworks capable of guiding suitable design of interactive systems in a particular setting. In the context of this study, and with ISCA need, the obtained HAIC framework was found to be appropriate. This study found out that the current user-centred design approach does not explain precisely how designers can employ activity-based interaction design techniques into the design process. Thus, the study contributes to the literature on users’ involvement in interactive systems’ design through the HADC framework provided

Special issue on robot and human interactive communication 2023 (selected papers from RO-MAN 2022)

Special issue on robot and human interactive communication 2023 (selected papers from RO-MAN 2022)

Towards Human Activity-Based Interactive Communication Systems’ Design in Higher Learning Institutions: Study Conducted in Tanzania Higher Learning Institutions

The power of human computer interaction in interactive systems’ design processes is unimaginable as it determines their usability. However, these design processes encounter several challenges which make most of the designed interactive systems, notably in higher learning institutions fail to suitably satisfy targeted users’ needs despite the fact that they are normally designed based on their requirements.
 This study begins by exploring the challenges facing main communication and interaction means used in Tanzania higher learning institutions. The study then provides a review of challenges related to key research areas associated with interactive systems’ design. Based on the reviewed challenges, authors combined design science research with activity theory to come up with suited techniques through which user-involved interactive communication frameworks needed for suitable design of human activity-based interactive communication systems’ can be obtained.
 Based on that approach, authors came up with a suited four phase Design Science Research methodology to be used in designing of applicable frameworks. In its first phase and following a crucial user-centred design process, authors were able to come up with a Human-Activity Design Centred Framework for capturing most of the users’ needs in the design process through activities performed as well as a Human Factors’ Approach to interactive communication systems design in HLIs’.

MoTiS Parameters for Expressive Multi-Robot Systems: Relative Motion, Timing, and Spacing

Multi-robot systems are moving into human spaces, such as working with people in factories (Bacula et al., in: Companion of the 2020 ACM/IEEE international conference on human–robot interaction, pp 119–121, 2020) or in emergency support (Wagner in Front Robot AI 8, 2021; Baxter et al., in: Autonomous robots and agents, Springer, pp 9–16, 2007) and it is crucial to consider how robots can communicate with the humans in the space. Our work evaluates a parameter framework to allow multi-robot groups of x, y, theta robots to effectively communicate using expressive motion. While expressive motion has been extensively studied in single robots (Knight et al., in: 2016 IEEE international conference on intelligent robots and systems (IROS), IEEE, 2016; Bacula and LaViers in Int J Soc Robot, 1–16, 2020; Dragan et al., in: 2013 8th ACM/IEEE international conference on human–robot interaction (HRI), IEEE, pp 301–308, 2013; Kirby et al., in: The 18th IEEE international symposium on robot and human interactive communication, 2009, RO-MAN 2009, IEEE, pp 607–612, 2009), moving to multi-robots creates new challenges as the state space expands and becomes more complex. We evaluate a hierarchical framework of six parameters to generate multi-robot expressive motion consisting of: (1) relative direction, (2) coherence, (3) relative speed, (4) relative start time, (5) proximity, and (6) geometry. We conducted six independent online studies to explore each parameter, finding that four out of six of the parameters had significant impact on people’s perception of the multi-robot group. Additional takeaways of our studies clarify what humans interpret as a robot group, when the group is perceived positively versus negatively, and the critical role of architectural floor plan in interpreting robot intent.

The 30th IEEE/RSJ International Conference on Robot and Human Interactive Communication (RoMAN 2021)

The 30th IEEE/RSJ International Conference on Robot and Human Interactive Communication (RoMAN 2021)

Towards Robots able to Measure in Real-time the Quality of Interaction in HRI Contexts

When humans interact with each other, collaborating on a shared activity or chatting, they are able to tell whether their interaction is going well or not and if they observe that its quality is deteriorating, they can adapt their behavior or invite their partner to act in order to improve it. A robot endowed with the ability to evaluate the quality of its interaction with its human partners, will have the opportunity to perform better since it will be better informed for its decision making processes. We propose metrics to be integrated in a cognitive and collaborative robot in order to measure in real-time the quality of an interaction (QoI). This permanent evaluation process has been implemented and tested within the high-level controller of an entertainment robot. A first demonstration shows the ability of the scheme to compute QoI for a direction-giving task and exhibit significant differences between its performance in interaction with a fully compliant human, a human confused by the course of action and a non-cooperative one. This paper is an extension and further refinement of work originally reported in Mayima (in: 29th IEEE International conference on robot and human interactive communication (RO-MAN), 2020).

Are friends electric? The benefits and risks of human-robot relationships.

SummarySocial robots that can interact and communicate with people are growing in popularity for use at home and in customer-service, education, and healthcare settings. Although growing evidence suggests that co-operative and emotionally aligned social robots could benefit users across the lifespan, controversy continues about the ethical implications of these devices and their potential harms. In this perspective, we explore this balance between benefit and risk through the lens of human-robot relationships. We review the definitions and purposes of social robots, explore their philosophical and psychological status, and relate research on human-human and human-animal relationships to the emerging literature on human-robot relationships. Advocating a relational rather than essentialist view, we consider the balance of benefits and harms that can arise from different types of relationship with social robots and conclude by considering the role of researchers in understanding the ethical and societal impacts of social robotics.

The Word is the Knife: Janus-Faced Communication in Sartre’s No Exit and Rose’s Twelve Angry Men

In this article the authors offer an analysis of Jean-Paul Sartre’s 1946 play Huis Clos (No Exit) and Reginald Rose’s 1954 play Twelve Angry Men, with particular attention paid to exploring the insights from each theatrical text about communication. The process of communication may be ambivalent or Janus-faced, and one of the objectives of this analysis is to consider communication in terms of its duality and incisive power. In doing so, the aim is to explore its antithetical tensions by amplifying the mythological, deliberative and philosophical dimensions of communication praxis. In particular, the archetype of the knife provides a useful metaphor for understanding the potentials and pitfalls of communication in human interaction. Scott Haden Church is an Assistant Professor in the School of Communications at Brigham Young University. He has recently published in Critical Studies in Media Communication. Jesse King Jones is in the Masters Programme of the School of Communications at Brigham Young University.

The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019)

The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019)

Teaching Robots a Lesson: Determinants of Robot Punishment

There have been multiple incidents where humans attacked robots in a public environment (Brscic et al., in: Proceedings of the international conference on human–robot interaction, ACM/IEEE, Portland, 2015, https://doi.org/10.1145/2696454.2696468 ); Vincent, in: A drunk man was arrested for knocking over Silicon Valley’s crime-fighting robot, 2017, https://www.theverge.com/2017/4/26/15432280/security-robot-knocked-over-drunk-man-knightscope-k5-mountain-view ; Mosbergen, in: Good job, America. You killed hitchBOT. Huffpost, 2015, https://www.huffpost.com/entry/hitchbot-destroyed-philadelphia_n_55bf24cde4b0b23e3ce32a67 ; Mutlu and Forlizzi, in: Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, ACM, 2008, https://doi.org/10.1145/2696454.2696468 ; Rehm and Krogsager, in: 2013 Proceedings of IEEE RO-MAN, IEEE, 2013, https://doi.org/10.1145/2696454.2696468 ; (Salvini et al., in: 19th International symposium in robot and human interactive communication, 2010). Although the form of aggression suggests that this behaviour might be motivated by the aggressor’s desire for social recognition rather than an urge for vandalism (Salvini et al. 2010; Keijsers and Bartneck, in: Proceedings of the international conference on human–robot interaction, ACM/IEEE, New York, 2018, https://doi.org/10.1145/2696454.2696468 ), very little is known about the underlying psychological mechanisms. Therefore, extending previous research, the current study investigated if human aggression towards a robot would be influenced by the aggressor’s feelings of power, the perception of the threat that robots in general might pose, mind attribution to the robot, and the robot’s embodiment. First, threat and power were manipulated. Subsequently, participants played a learning task with either a virtual or an embodied robot. Mind attribution was measured afterwards. Participants were asked to restrict the robot’s energy supply after each wrong answer, which was taken as a measure of aggression. Results indicated that an embodied robot was punished less harshly than a virtual one, except for when people had been primed with power and threat. Being primed with power diminished the influence of mind attribution. Mind attribution increased aggression in the threat condition but was related to decreased aggression when people had not been reminded of threat.

Special issue on robot and human interactive communication

We are pleased to announce the special issue on Robot and Human Interactive Communication 2020. The first special issue was published in 2019 to celebrate the promotion of the IEEE International Sy...

Special issue on robot and human interactive communication

We are pleased to announce the special issue on Robot and Human Interactive Communication 2020. The first special issue was published in 2019 to celebrate the promotion of the IEEE International Sy...

Effectiveness of Storytelling in Agricultural Marketing: Scale Development and Model Evaluation.

Storytelling is a mode of communication in human interaction and is pervasive in everyday life. Storytelling in marketing is also a managerial application as a marketing strategy. Researchers of consumer psychology and marketing have devoted great efforts to developing theories and conducting empirical studies on this approach. However, in addition to narrative theories, many researchers are mainly concerned about the effect of telling a good brand story and its applications, such as advertising design and presentation. However, for those products that usually lacks branding, such as agricultural products, knowledge remains scarce about the relative impact of storytelling in marketing. Few researchers have explicitly developed a valid tool for measuring the effect of storytelling in marketing. To aid storytelling research in consumer psychology, this article conceptualized a construct of the effectiveness of storytelling in agricultural marketing and developed a measure with further validation. This scale consisted of 13 items with four subscales: narrative processing, affect, brand attitude, and purchase intention. The findings of this study supported a structural model with strong order among the four dimensions and good model fit. A discussion of the results and the theoretical and practical implications for consumer psychology and marketing practice are also addressed.

Ostensive-Cue Sensitive Learning and Exclusive Evaluation of Policies: A Solution for Measuring Contingency of Experiences for Social Developmental Robot.

Joint attention related behaviors (JARBs) are some of the most important and basic cognitive functions for establishing successful communication in human interaction. It is learned gradually during the infant's developmental process, and enables the infant to purposefully improve his/her interaction with the others. To adopt such a developmental process for building an adaptive and social robot, previous studies proposed several contingency evaluation methods, by which an infant robot becomes able to sequentially learn some primary social skills. These skills included gaze following and social referencing, and could be acquired through interacting with a human caregiver model in a computer simulation. However, to implement such methods to a real-world robot, two major problems, that were not addressed in the previous research, have remained unresearched: (1) dependency of histogram of the observed events by the robot to each other, which increases the error of the internal calculation and consequently decreases the accuracy of contingency evaluation; and (2) unsynchronized teaching/learning phase of the teaching-caregiver and the learning-robot, which leads the robot and the caregiver not to understand the suitable timing for the learning and the teaching, respectively. In this paper, we address these two problems, and propose two algorithms in order to solve them: (1) exclusive evaluation of policies (XEP) for the former, and (2) ostensive-cue sensitive learning (OsL) for the latter. To show the effect of the proposed algorithms, we conducted a real-world human-robot interaction experiment with 48 subjects, and compared the performance of the learning robot with/without proposed algorithms. Our results show that adopting proposed algorithms improves the robot's performance in terms of learning efficiency, complexity of the learned behaviors, predictability of the robot, and even the result of the subjective evaluation of the participants about the intelligence of the robot as well as the quality of the interaction.

The Effects of Humanlike and Robot-Specific Affective Nonverbal Behavior on Perception, Emotion, and Behavior

Research demonstrated that humans are able to interpret humanlike (affective) nonverbal behavior (HNB) in artificial entities (e.g. Beck et al., in: Proceedings of the 19th IEEE international symposium on robot and human interactive communication, IEEE Press, Piscataway, 2010. https://doi.org/10.1109/ROMAN.2010.5598649 ; Bente et al. in J Nonverbal Behav 25: 151–166, 2001; Mumm and Mutlu, in: Proceedings of the 6th international conference on human–robot interaction, HRI. ACM Press, New York, 2011. https://doi.org/10.1145/1957656.1957786 ). However, some robots lack the possibility to produce HNB. Using robot-specific nonverbal behavior (RNB) such as different eye colors to convey emotional meaning might be a fruitful mechanism to enhance HRI experiences, but it is unclear whether RNB is as effective as HNB. We present a review on affective nonverbal behaviors in robots and an experimental study. We experimentally tested the influence of HNB and RNB (colored LEDs) on users’ perception of the robot (e.g. likeability, animacy), their emotional experience, and self-disclosure. In a between-subjects design, users ( $$n=80$$ ) interacted with either (a) a robot displaying no nonverbal behavior, (b) a robot displaying affective RNB, (c) a robot displaying affective HNB or (d) a robot displaying affective HNB and RNB. Results show that HNB, but not RNB, has a significant effect on the perceived animacy of the robot, participants’ emotional state, and self-disclosure. However, RNB still slightly influenced participants’ perception, emotion, and behavior: Planned contrasts revealed having any type of nonverbal behavior significantly increased perceived animacy, positive affect, and self-disclosure. Moreover, observed linear trends indicate that the effects increased with the addition of nonverbal behaviors (control< RNB< HNB). In combination, our results suggest that HNB is more effective in transporting the robot’s communicative message than RNB.

The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017)

The 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2017)

Recent development of speech and audio signal processing in network communication

Speech and audio signal processing plays an important role in human interactive communication over networks. With the development of wide-band and mobile networks, humans are w illing to exchange more fruitful and fluent speech and audio contents over the networks. These developments advance the techniques of speech and audio signal processing including systems, theories, algorithms and processing tools.