Lightweight Interaction Research Articles

Enabling lightweight immersive user interaction in smart buildings through learning-based mobile panorama streaming

Smart buildings integrate users’ digitized wearables with their physical surroundings, creating a seamless and interactive user experience. This is achieved through the utilization of multiple sensors, video streaming, artificial intelligence, and edge computing. These technologies gather extensive data and provide users with a wide range of applications, such as 3D audio/video in AR/VR, localization, virtual tours, and vigilant monitoring. Nevertheless, the current AR/VR devices face limitations due to the bulkiness and discomfort of the hardware used for on-body sensing, such as headsets and specialized glasses. These components often become uncomfortable during prolonged usage, posing a challenge for creating an immersive system that combines lightweight interaction with high-quality presentation.This paper presents a comprehensive system designed to enable immersive interaction in smart buildings with a focus on lightweight solutions. The system consists of the following components: (1). A lightweight panoramic imaging framework to address the challenges related to hardware size and functionality. (2). A learning-based video transcoding cost prediction framework for efficient load balancing. (3). A layered networking architecture designed to facilitate high-quality mobile panorama live streaming. Collectively, these components offer lightweight interaction paired with enhanced presentation quality. Our experimental results demonstrate the effectiveness of the system design, showcasing its seamless operation across different times, geographical locations, and heterogeneous wireless networks.

Cross-modal learning with multi-modal model for video action recognition based on adaptive weight training

The canonical video action recognition methods usually label categories with numbers or one-hot vectors and train neural networks to classify a fixed set of predefined categories, thereby constraining their ability to recognise complex actions and transferable ability to unseen concepts. In contrast, cross-modal learning can improve the performance of individual modalities. Based on the facts that a better action recogniser can be built by reading the statements used to describe actions, we exploited the recent multimodal foundation model CLIP for action recognition. In this study, an effective Vision-Language action recognition adaptation was implemented based on few-shot examples spanning different modalities. We added semantic information to action categories by treating textual and visual label as training examples for action classifier construction rather than simply labelling them with numbers. Due to the different importance of words in text and video frames, simply averaging all sequential features may result in ignoring keywords or key video frames. To capture sequential and hierarchical representation, a weighted token-wise interaction mechanism was employed to exploit the pair-wise correlations adaptively. Extensive experiments with public datasets show that cross-modal action recognition learning helps for downstream action images classification, in other words, the proposed method can train better action classifiers by reading the sentences describing action itself. The method proposed in this study not only reaches good generalisation and zero-shot/few-shot transfer ability on Out of Distribution (OOD) test sets, but also performs lower computational complexity due to the lightweight interaction mechanism with 84.15% Top-1 accuracy on the Kinetics-400.

Design of a SMA-based soft composite structure for wearable rehabilitation gloves.

The combination of smart soft composite structure based shape memory alloy (SMA) and exoskeleton technology has the advantages of light weight, energy saving, and great human-exoskeleton interaction. However, there are no relevant studies on the application of SMA-based soft composite structure (SSCS) in hand exoskeletons. The main difficulty is that directional mechanical properties of SSCS need to comply with fingers movement, and SSCS can deliver enough output torque and displacement to the relevant joints. This paper aims to study the application of SSCS for wearable rehabilitation gloves and explore its bionic driving mechanism. This paper proposes a soft wearable glove (Glove-SSCS) for hand rehabilitation actuated by the SSCS, based on finger force analysis under different drive modes. The Glove-SSCS can support five-finger flexion and extension, weighs only 120 g, and adopts modular design. Each drive module adopts a soft composite structure. And the structure integrates actuation, sensing and execution, including an active layer (SMA spring), a passive layer (manganese steel sheet), a sensing layer (bending sensor) and connection layers. To obtain a high-performance SMA actuators, the performance of SMA materials was tested in terms of temperature and voltage, temperature at the shortest length, pre-tensile length and load. And the human-exoskeleton coupling model of Glove-SSCS is established and analyzed from force and motion. The results show that the Glove-SSCS can realize bidirectional movements of fingers flexion and extension, with ranges of motion are 90-110° and 30-40°, and their cycles are 13-19 s and 11-13 s. During the use of Glove-SSCS, the temperature of gloves is from 25 to 67°C, and the surface temperature of hands is from 32 to 36°C. The temperature of Glove-SSCS can be kept at the lowest temperature of SMA operation without much impact on the human body.

A commonality-based enhancement for sentence modeling with supervision

Sentence pair modeling is a fundamental yet challenging issue for feature mining in natural language processing (NLP) tasks. Recently, most works have generated feature and sentence representation based on the interactive attention mechanism. However, these models have two limitations: (1) they only consider global information through attention coefficient weighting, which makes insufficient utilization of critical features; (2) they only conduct internal training by fine-tuning network parameters, in which attention results are poorly explained. In this paper, inspired by human reasoning, we propose a Commonality Aggregated approach (CA) to enhance the lightweight interaction model by considering phrase features and contextual words. Specifically, we first fuse positional encoding and employ supervised training to extract critical phrase information from the text as the commonality of sentence pairs. Then, we deploy transfer learning and utilize interaction network to combine crucial phrase features, core word features, and positional encoding to enhance sentence pair modeling. Compared with the original network, extensive experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed commonality aggregated method with stronger competitiveness. Further visual analysisanalysies validated the more explicit interpretability of attention, and extended experimental results indicate the excellent generalization of our approach.

BIM Collaborative Design Solutions for Super High-Rise Buildings Over 250 Meters

Super high-rise buildings are the product of human economic activities and technological progress. Construction of super high-rise buildings is difficult and involves many participants. Super high-rise buildings over 250 meters are facing more implementation difficulties, and design stage is the initial stage of the construction process. How to promote the design work of all participants in the design phase is a problem that must be solved in the design of super high-rise buildings. This research aims at 4 special difficulties in the design of super high-rise buildings with a height of 250 meters: multi-specialty collaborative design and management, Multi-party collaborative design and management, lightweight interaction of BIM models, data format exchange between software. This study adopts an integrated design platform to propose corresponding solutions to promote the collaborative design of super high-rise buildings. Finally, a super high-rise building design project is used as a case to demonstrate the research method.

Cultural Heritage ‘In-The-Wild’

As digital cultural heritage applications begin to be deployed outwith ‘traditional’ heritage sites (such as museums, open-air museums or galleries), there is an increased need to consider their use amongst individuals who are open to learning about the heritage of a site, but where that is a clearly secondary purpose of their visit. Parks, recreational areas and the everyday built environment represent places that although often rich in heritage, are often not visited primarily to access that heritage. We present the results of a study of a mobile application, called Explore, that supports accessing heritage on a Finnish recreational island. Evaluation with 45 participants, who were not visiting primarily to access the heritage, provided insight into how digital heritage applications can be developed for this user group. Our results showed how low immersion and lightweight interaction support individuals to integrate cultural heritage around their primary visit purpose. Although participants were willing to include heritage as part of their visit, they were not willing to be directed by Explore. Our work outlines future directions that should be considered when expanding the reach of heritage access beyond ‘traditional’ sites.

Designing for Digital Reading

Reading is a complex human activity that has evolved, and co-evolved, with technology over thousands of years. Mass in the fifteenth century firmly established what we know as the modern book, with its physical format of covers and paper pages, and now-standard features such as page numbers, footnotes, and diagrams. Today, electronic documents are enabling paperless reading supported by eReading technologies such as Kindles and Nooks, yet a high proportion of users still opt to print on paper before reading. This persistent habit of printing to read is one sign of the shortcomings of digital documents -- although the popularity of eReaders is one sign of the shortcomings of paper. How do we get the best of both worlds? The physical properties of paper (for example, it is light, thin, and flexible) contribute to the ease with which physical documents are manipulated; but these properties have a completely different set of affordances to their digital equivalents. Paper can be folded, ripped, or scribbled on almost subconsciously -- activities that require significant cognitive attention in their digital form, if they are even possible. The nearly subliminal interaction that comes from years of learned behavior with paper has been described as lightweight interaction, which is achieved when a person actively reads an article in a way that is so easy and unselfconscious that they are not apt to remember their actions later. Reading is now in a period of rapid change, and digital text is fast becoming the predominant mode of reading. As a society, we are merely at the start of the journey of designing truly effective tools for handling digital text. This book investigates the advantages of paper, how the affordances of paper can be realized in digital form, and what forms best support lightweight interaction for active reading. To understand how to design for the future, we review the ways reading technology and reader behavior have both changed and remained constant over hundreds of years. We explore the reasoning behind reader behavior and introduce and evaluate several user interface designs that implement these lightweight properties familiar from our everyday use of paper. We start by looking back, reviewing the development of reading technology and the progress of research on reading over many years. Drawing key concepts from this review, we move forward to develop and test methods for creating new and more effective interactions for supporting digital reading. Finally, we lay down a set of lightweight attributes which can be used as evidence-based guidelines to improve the usability of future digital reading technologies. By the end of this book, then, we hope you will be equipped to critique the present state of digital reading, and to better design and evaluate new interaction styles and technologies.

Extending the Interaction Nets Calculus by Generic Rules

We extend the textual calculus for interaction nets by generic rules and propose constraints to preserve uniform confluence. Furthermore, we discuss the implementation of generic rules in the language inets, which is based on the lightweight interaction nets calculus.

PivotPaths: Strolling through Faceted Information Spaces.

We present PivotPaths, an interactive visualization for exploring faceted information resources. During both work and leisure, we increasingly interact with information spaces that contain multiple facets and relations, such as authors, keywords, and citations of academic publications, or actors and genres of movies. To navigate these interlinked resources today, one typically selects items from facet lists resulting in abrupt changes from one subset of data to another. While filtering is useful to retrieve results matching specific criteria, it can be difficult to see how facets and items relate and to comprehend the effect of filter operations. In contrast, the PivotPaths interface exposes faceted relations as visual paths in arrangements that invite the viewer to `take a stroll' through an information space. PivotPaths supports pivot operations as lightweight interaction techniques that trigger gradual transitions between views. We designed the interface to allow for casual traversal of large collections in an aesthetically pleasing manner that encourages exploration and serendipitous discoveries. This paper shares the findings from our iterative design-and-evaluation process that included semi-structured interviews and a two-week deployment of PivotPaths applied to a large database of academic publications.

MultiNet

This demo presents MultiNet , a novel method for joining devices to a domestic Wi-Fi network. MultiNet dynamically reconfigures the network to accept each device, rather than configuring each device to fit the network as is the norm. It does so by assuming that each device is pre-configured with a cryptographically generated WPA2 network SSID/passphrase pair, and then providing a lightweight interaction through which the user creates a new network for each device. This approach makes securely adding devices to a wireless network straightforward without compromising security or burdening the user, and maintaining backward compatibility with existing deployed standards and protocols. The demo deploys a MultiNet Access Point (AP) and a number of Wi-Fi enabled consumer devices to allow viewers to dynamically construct and deconstruct the network via the MultiNet controller currently implemented as an app on an Android phone (Figure 1). The code for MultiNet is publicly available under open-source licenses.

Adopt, adapt, abandon: Understanding why some young adults start, and then stop, using instant messaging

Instant messaging (IM) has become a popular and important mode of staying in touch for teens and young adults. It allows for easy, frequent and lightweight interaction that contributes to building and sustaining friendships, as well as coordinating social activities. Despite the initial appeal of IM, however, some have found it too distracting and have changed their usage or abandoned it. I interviewed 21 former users of IM about their adoption, usage and eventual abandonment of the technology. Results show that participants were initially attracted to features of IM that enabled them to maximize their use of leisure time via easy and frequent interaction with their friends, but that, in a different usage context, these same features became distracting and annoying. Participants adapted their behavior to avoid these drawbacks, but IM did not support these adaptations effectively. In particular, IM did not allow for control over interruptions, which became more important as their contact lists grew and social time became scarce; and they ultimately abandoned the technology. These results point to a need for understanding use beyond adoption, and a theoretical and practical focus on understanding the adaptation and changing utility that accompany long-term usage of technologies.

Cognitive Ergonomics in Virtual Environments: Development of an Intuitive and Appropriate Input Device for Navigating in a Virtual Maze

For patients suffering from mild cognitive impairments, the navigation through a virtual maze should be as intuitive and efficient as possible in order to minimize cognitive and physical strain. This paper discusses the appropriateness of interaction devices for being used for easy navigation tasks. Information gained from human centered evaluation was used to develop an intuitive and ergonomic interaction device. Two experiments examined the usability of tracked interaction devices. Usability problems with the devices are discussed. The findings from the experiments were translated into general design guidance, in addition to specific recommendations. A new device was designed on the basis of these recommendations and its usability was evaluated in a second experiment. The results were used to develop a lightweight interaction device for navigation in the virtual maze.

TeleNotes

Communication theories and technology have tended to focus on extended, formal meetings and have neglected a prevalent and vital form of workplace communication—namely, lightweight communication. Unlike formal, extended meetings, lightweight interaction is brief, informal, unplanned, and intermittent. We analyze naturalistic data from a study of work-place communication and derive five design criteria for lightweight interaction systems. These criteria require that systems for lightweight interaction support conversational tracking, rapid connection , the ability to leave a message, context management, and shared real-time objects . Using these criteria, we evaluate existing interpersonal communications technologies. We then describe an implementation of a system (TeleNotes) that is designed to support lightweight interaction by meeting these criteria. The interface metaphor allows communications to be based around desktop objects, resembling “sticky notes.” These objects are also organized into “desktop piles” to support conversational threads and provide mechanisms for initiating real-time audio, video, and application sharing. We conducted informal user testing of several system prototypes. Based on our findings, outstanding issues concerning theory and systems design for communication systems are outlined—in particular, with regard to the issue of managing conversations over time.