Touchscreen Devices Research Articles

SwipeCut: Interactive Segmentation via Seed Grouping

Interactive image segmentation algorithms rely on the user to provide annotations as the guidance. When the task of interactive segmentation is performed on a small touchscreen device, the requirement of providing precise annotations could be cumbersome to the user. We design a new interaction mechanism that actively queries seeds for guiding the user to label. Our method enforces sparsity and diversity criteria on the selection of query seeds, and at each round of interaction, the user is only presented with a small number of informative query seeds that are far apart from each other. Therefore, the user merely has to swipe through on the ROI-relevant query seeds for checking which ones of the query seeds are inside the region of interest (ROI). This kind of interaction should be easy since those gestures are commonly used on a touchscreen. As a result, we are able to derive a user-friendly interaction mechanism for annotation on small touchscreen devices. The performance of our algorithm is evaluated on six publicly available datasets. The evaluation results show that our algorithm achieves high segmentation accuracy, with short computational time and less user feedback.

Hogyan hat a mobileszköz-használat az óvodások figyelmére és társas-kognitív készségeire?

Az érintőképernyős mobileszközöket (ÉKM: okostelefonok és tabletek) egyre többet és egyre fiatalabb korban használják a gyerekek. Ekkor az agy még nagyon plasztikus, így az ÉKM-használat befolyásolhatja a kognitív fejlődést, ami új kihívásokat támaszt a szülőkkel és pedagógusokkal szemben. Egy keresztmetszeti vizsgálatban összehasonlítottuk az ÉKM-eszközöket gyakran használó és az ilyen eszközöket nem használó óvodás korú gyerekek (szelektív és megosztott) figyelmi és szociokognitív képességeit. Majd kísérletesen vizsgáltuk, hogy miképp befolyásolja a gyerekek figyelmi teljesítményét, ha előzőleg (előkezelés) egy gyors vagy lassú sebességű digitális játékkal vagy egy nem digitális játékkal játszanak. A szelektív figyelmi feladatban a gyerekek figyelmére (ÉKM-használattól függetlenül) a globális fókusz volt jellemző (jobban teljesítettek, ha az egészre kellett figyelni, mint ha a részletre), a megosztott figyelmi feladatban azonban az ÉKM-használók atipikus, lokális fókuszt mutattak (jobban teljesítettek, amikor a kis ábra volt a célinger, mint amikor a nagy), és teljesítményük a globális próbákban elmaradt az ÉKM-eket nem használó gyerekekétől. Az ÉKM-használók gyengébb tudatelméleti teljesítményt mutattak, viszont az érzelmek felismerésében nem különböztek az ÉKM-et nem használóktól. Az előkezelésen alapuló vizsgálatunk a figyelmi működéssel kapcsolatos eredményeket kísérletesen is megerősítette: a tesztet megelőzően digitális játékkal játszó gyerekek globális fókuszt mutattak a szelektív feladatban és lokális fókuszt a megosztott figyelmi feladatban, a nem-digitális játék után viszont mindkét feladatban globális fókusz volt jellemző. Míg a nem-digitális és a lassú digitális játékkal játszó gyerekek a szelektív figyelmi feladatban jobban teljesítettek, mint a megosztottban, addig a gyors digitális játékkal játszók nem mutattak ilyen előnyt a szelektív figyelmi feladatban. Eredményeink azt mutatják, hogy az ÉKM-eszközök rövid- és hosszútávú használata és azon belül a digitális játékokkal való játék lokális figyelmi fókuszhoz vezet, talán azért, mert a digitális képernyők lokális információkban gazdagok, és ritkán látszódik egyben az egész kép. A gyors digitális játék, azáltal, hogy párhuzamosan több ingerre kell figyelni, a megosztott figyelmet fejleszti, de a szelektív figyelmet nem. Az ÉKM-használat elveszi az időt a társas tevékenységektől, ami magyarázhatja, hogy az ÉKM-használóknak nehézségeik vannak a komplexebb szociokognitív képességek terén (tudatelmélet). Eredményeink fontos ismereteket nyújtanak és irányt mutatnak a pedagógia és a pszichológia számára.

Experimental Evaluation on the Effect of Electrode Configuration in Electrostatic Actuators for Increasing Vibrotactile Feedback Intensity

Vibrotactile feedback is a key feature of many modern touch displays, which greatly enhances user experiences when interacting with an onscreen interface. Despite its popularity in small touch screen devices, this haptic feature is absent in most large displays due to a lack of suitable actuators for such applications. Thus, a growing need exists for haptic actuators capable of producing sufficient vibrations in large touch displays. This study proposes and evaluates a novel electrostatic resonant actuator (ERA) with a moving mass and dual electrodes for increased vibration feedback intensity. The dual-electrode ERA was fabricated along with a comparable single-electrode ERA to investigate the effect of the electrode configuration on the maximum vibration intensity. When measured directly on the mass, the maximum vibration intensity of the dual-electrode actuator increased by 73% compared to the single-electrode actuator. When mounted and measured on a mock panel, the maximum vibration intensity of the dual-electrode actuator increased by nearly 65% compared to a similarly mounted single-electrode actuator. These results show that the dual-electrode configuration can significantly increase the vibration intensity when compared to the conventional ERA. This demonstrates a promising potential for the use of the proposed actuator for generating vibrotactile feedback in large touch displays.

Systematic Review of Behavioral Interventions Using Digital Technology to Reduce Problem Behavior in the Classroom

Problem behavior in the classroom leads to challenges for teachers who often feel ill equipped to address such behavior. Behavior analytic interventions have been shown to be effective for reducing problem behavior in the classroom. Less is known about the impact of these interventions when implemented using digital technology. The purpose of the current systematic review was to synthesize and evaluate the quality of the literature. We identified 19 studies which demonstrated the effectiveness of these interventions in both general education and special education classrooms. Across these studies, touch-screen devices and computers were the most frequently used form of digital technology. Implications for practice and future research will be discussed.

Tactile discrimination of material properties: application to virtual buttons for professional appliances

An experiment is described that tested the possibility to classify wooden, plastic, and metallic objects based on reproduced auditory and vibrotactile stimuli. The results show that recognition rates are considerably above chance level with either unimodal auditory or vibrotactile feedback. Supported by those findings, the possibility to render virtual buttons for professional appliances with different tactile properties was tested. To this end, a touchscreen device was provided with various types of vibrotactile feedback in response to the sensed pressing force and location of a finger. Different virtual buttons designs were tested by user panels who performed a subjective evaluation on perceived tactile properties and materials. In a first implementation, virtual buttons were designed reproducing the vibration recordings of real materials used in the classification experiment: mainly due to hardware limitations of our prototype and the consequent impossibility to render complex vibratory signals, this approach did not prove successful. A second implementation was then optimized for the device capabilities, moreover introducing surface compliance effects and button release cues: the new design led to generally high quality ratings, clear discrimination of different buttons and unambiguous material classification. The lesson learned was that various material and physical properties of virtual buttons can be successfully rendered by characteristic frequency and decay cues if correctly reproduced by the device.

User Identification Based on the Dynamic Features Extracted from Handwriting on Touchscreen Devices

<p><strong>Abstract</strong> - <strong>This research presents a methodology for user identification using ten English words written by a finger on smartphone and mini-tablet. This research considers three features, namely Signature Precision (SP), Finger Pressure (FP), and Movement Time (MT) that were extracted from each of ten English words using dynamic time warping. The features are then used individually and combined for the purpose of user identification based on the Euclidean distance and the k-nearest neighbor classifier. We concluded that the best identification accuracy results from the combinations of (SP and FP) features with an average accuracies of 74.55% and 69% were achieved on small smartphone and Mini-tablet respectively using a dataset of 42 users.</strong></p>

User-Silicon Entangled Mobile Identity Authentication

We explore mobile device touchscreen characteristics to build a user-device (UD) biometric physical unclonable function (PUF). Human user touchscreen interaction induces dynamic capacitive differences. Sensors detect current differences which are a function of both (1) a human biometric of how a shape is traced and (2) silicon foundry process transistor-level variability embedded in the touchscreen grid. This forms a physical function with input x defining a shape and output y abstracted from the measured current value stream. We argue and establish that this physical function has PUF attributes. Moreover, it provides a robust user-device biometric-based authentication mechanism. Authentication is based on geometric shapes (challenges) drawn on the touchscreen. Users trace them. The authentication layer creates a response abstract, and validates it against a user profile. Authentication accuracy is affected by the complexity of geometric shapes as well as the validation algorithm. We consider polyline shapes (simple gestures) and complex closed geometric shapes (complex gestures). Complex gestures offer higher response entropy, but are computationally less efficient with a slightly lower validation accuracy. Complex gestures achieve 99.6% accuracy compared with 100% for simple gestures. User profiles exhibit physical unclonable function (PUF) properties. Touchscreen gestures are quantized into binary strings. Gesture hamming distance is 60+ bits for 128-bit strings for different user-device profiles; it is 0 bits for the same profile. This demonstrates variability and reproducibility respectively. Montreal TestU01 tests binary string pseudorandom characteristics; the majority of tests pass showing pseudorandom number generator (PRG) characteristics.

TimelyRep: Timing deterministic replay for Android web applications

SummaryWith the constantly growing and changing requirements of app users, web techniques are used in mobile application development for better cross‐platform compatibility and online update. As the embedded web contents gain complexity, debugging web apps become a critical demand. Web replay tools can record program inputs and reproduce the same execution for debugging and performance tuning. However, traditional replay approaches are largely intended for apps with desktop interaction methods (keyboard, mouse) and require modification to the browser, which limits their applicability in mobile platforms.In this paper, we develop TimelyRep, which provides deterministic record‐and‐replay as a software library, running on commodity Android. TimelyRep can be used for app development with unmodified Android devices and for production to collect faulty execution from users. Also, we propose an efficient replay timing control mechanism and achieve higher timing precision as facing higher event rate on touchscreen devices. TimelyRep also supports cross‐device replay and can replay logged event traces on different devices, which is useful for developers to reproduce user inputs on their own devices. We evaluate TimelyRep with real‐world web applications. The results show that TimelyRep is useful for recreating program bugs and maintaining low delays for touch‐intensive web games.

Touch vs. click: how computer interfaces polarize consumers’ evaluations

Increasingly powerful computer technologies have enabled the development and widespread growth of touchscreen devices such as computers, tablets, and smartphones in consumers’ daily lives, including online shopping. Nonetheless, it is not clear to what extent and how direct-touch (i.e., finger) vs. indirect-touch (e.g., mouse click or stylus) interfaces have differential effects on consumers’ evaluations toward the object on the screen. Results from a lab experiment and a field study indicate that a direct-touch (vs. indirect-touch) interface has a polarizing effect on consumer evaluations. For an object about which consumers have a prior positive attitude, a direct-touch interface enhances consumer evaluations; for an object about which consumers have a prior negative attitude, a direct-touch interface lowers consumer evaluations. We find that consumers’ visual information processing style can moderate the polarizing effect. In addition, the polarizing effect can be explained by consumers’ vividness perception. These findings make useful contributions to the literature on haptic effects and human-computer interactions, as well as have significant managerial implications.

The use of technology in the management of children with phonological delay and adults with acquired dysarthria: A UK survey of current speech-language pathology practice

Purpose Technology is increasingly important for the speech-language pathology profession, but little is currently known about its use by clinicians. This study aimed to determine (i) the types of technology that speech-language pathologists (SLPs) in the UK have access to and use in practice and (ii) the barriers they encounter when assessing and treating adults with acquired dysarthria and children with phonological delay. Method UK SLPs were invited to complete two online surveys covering device availability, the use of technology for the assessment and treatment of acquired dysarthria and phonological delay, and barriers to using technology. Results were analysed using descriptive statistics. Result 126 SLPs completed the surveys. Most respondents had a range of devices available in clinic, including computer and touchscreen devices. Technology was primarily used for treatment to engage clients, provide direct feedback in sessions and encourage home practice. Reported key barriers include lack of knowledge and training, and technical support issues. Conclusion The use of technology in UK clinical practice varies widely, and technology adoption is hampered by various barriers. Findings indicate a need for more collaborative work between SLPs, technologists and policy-makers to develop the evidence-base for technology use in the management of acquired dysarthria and phonological delay.

Isolation of Gram-Positive, Antibiotic-Resistant Bacteria from Tactile Mobile Phones in a Northwestern Mexican City.

The widespread use of touch-screen mobile devices renders them potential fomites for the transmission of bacterial pathogens among users of different ages. The objectives of the present research were to isolate bacteria from mobile phones, perform molecular and phylogenetic identification, and determine the antibiotic resistance profiles. The surfaces of 50 touch-screen mobile devices owned by bystanders were sampled in the city center of Culiacan, Sinaloa, Mexico. The samples were cultured on nutritive agar; 13 bacterial colonies were isolated and characterized based on their macroscopic and microscopic characteristics and then identified using PCR amplification and sequencing of the 16S rRNA gene V4 and V6 regions. Their taxonomic relationships were determined via a Bayesian inference approach. Antimicrobial resistance was evaluated via disc diffusion and broth microdilution assays. Species of the genera Staphylococcus, Bacillus, and Enterococcus were identified on 84.6, 7.7, and 7.7% of the mobile phones, respectively. A unique subgroup of Staphylococcus epidermidis was identified in strains FBOPL-23, CAEPL-28, and FREPL-28. Staphylococcus hominis novobiosepticus was also identified on mobile phones for the first time. Of the isolated bacteria, 92.3% were resistant to erythromycin, 76.9% to ampicillin and penicillin, 61.5% to dicloxacillin, 38.5% to cephalothin and 7.7% to cefotaxime and ceftriaxone. The presence of antibiotic-resistant bacteria of clinical relevance poses potential risks to users' health and the dissemination of antibiotic resistance mechanisms throughout the community; thus, we recommend regular cleaning to prevent cross-infection by multidrug-resistant bacteria when using touch-screen mobile devices.

Improving electrical conductivity of leather surface: a new technology versus industrial applications

The present study takes into account the consumer requirement of an even more massive use of touch screen devices, where the leather has to compete with the modern textile fibers in terms of technological performances, with particular reference to the newest technologies for the production of conductive gloves and other kinds of conductive goods, like shoes, garments, and others. Most of the known technologies able to provide materials with a unique ability to discharge static electricity and conductivity comparable with the human body concern the use of textile fibers. On the other hand, the goal of the present work is to delve into the application of technologies for leather matrix, with particular reference to the use of carbon-based nanomaterials, and attention to economical and/or practical aspects.

Design Guidelines for Schematizing and Rendering Haptically Perceivable Graphical Elements on Touchscreen Devices

ABSTRACT This paper explores the viability of new touchscreen-based haptic/vibrotactile interactions as a primary modality for perceiving visual graphical elements in eyes-free situations. For touchscreen-based haptic information extraction to be both accurate and meaningful, the onscreen graphical elements should be schematized and downsampled to: (1) maximize the perceptual specificity of touch-based sensing and (2) account for the technical characteristics of touchscreen interfaces. To this end, six human behavioral studies were conducted with 64 blind and 105 blindfolded-sighted participants. Experiments 1–3 evaluated three key rendering parameters that are necessary for supporting touchscreen-based vibrotactile perception of graphical information, with results providing empirical guidance on both minimally detectable and functionally discriminable line widths, inter-line spacing, and angular separation that should be maintained. Experiments 4–6 evaluated perceptually-motivated design guidelines governing visual-to-vibrotactile schematization required for tasks involving information extraction, learning, and cognition of multi-line paths (e.g., transit-maps and corridor-intersections), with results providing clear guidance as to the stimulus parameters maximizing accuracy and temporal performance. The six empirically-validated guidelines presented here, based on results from 169 participants, provide designers and content providers with much-needed guidance on effectively incorporating perceptually-salient touchscreen-based haptic feedback as a primary interaction style for interfaces supporting nonvisual and eyes-free information access.

The Interplay Between the Natural Number Bias and Fraction Magnitude Processing in Low-Achieving Students

Research has identified two core difficulties many students have with fractions: First, they often struggle with processing fraction magnitudes, and second, they rely on natural number concepts in fraction problems (“Natural Number Bias”, NNB). Yet, the relation between these two difficulties is not well understood. Moreover, while most studies of the NNB relied on analyses of the whole samples, there is empirical evidence that the occurrence of the NNB differs between student subgroups. In the present study, we investigate individual students’ profiles of the occurrence of the natural number bias and their ability to process fraction magnitude, using a dynamic assessment that utilizes continuous diagrams on touchscreen devices. We analyze data of 234 low-achieving 6th-grade students from Germany who completed a symbolic fraction comparison task, and a fraction magnitude estimation task with continuous circle and tape diagrams. A cluster analysis on the comparison task revealed three distinct clusters: a Typical Bias cluster (better performance on symbolic fraction comparison items congruent to natural number-based reasoning), a Reverse Bias cluster (better performance on items incongruent to natural number-based reasoning), and a No Bias cluster (similar performance on congruent and incongruent items). Only students in the No Bias cluster but not students in the other clusters demonstrated a distance effect in symbolic fraction comparison, suggesting fraction magnitude processing. Linear mixed models on the percent absolute error in the magnitude estimation task revealed significantly lower percent absolute error for students in the No Bias cluster compared to students in the other two clusters. Moreover, students in the No Bias cluster were significantly slower to solve both fraction comparison and fraction magnitude estimation tasks than students in the other clusters. These results suggest that the occurrence of the natural number bias and the ability to process fraction magnitude are closely related. The continuous representations used in our digital assessment tools appeared to be suitable for assessing both the natural number bias and fraction magnitude processing.

Layout Transposition for Non-Visual Navigation of Web Pages by Tactile Feedback on Mobile Devices.

In this paper, we present the results of an empirical study that aims to evaluate the performance of sighted and blind people to discriminate web page structures using vibrotactile feedback. The proposed visuo-tactile substitution system is based on a portable and economical solution that can be used in noisy and public environments. It converts the visual structures of web pages into tactile landscapes that can be explored on any mobile touchscreen device. The light contrasts overflown by the fingers are dynamically captured, sent to a micro-controller, translated into vibrating patterns that vary in intensity, frequency and temperature, and then reproduced by our actuators on the skin at the location defined by the user. The performance of the proposed system is measured in terms of perception of frequency and intensity thresholds and qualitative understanding of the shapes displayed.

Immersion through Google Street View: multimodal representation of storyworld in Entrances & Exits by Reif Larsen

This article investigates the potential of new media for the creation of immersive verbal-visual forms by using Reif Larsen’s Entrances & Exits (2016) as a case study. Designed for touchscreen devices such as tablets and mobile phones, this digital imagetext combines Google Street View and verbal passages to produce a narrative with which the reader is invited to engage in a haptic manner, going beyond the mode of reception typical of print literature. Having discussed the hybridity of Entrances & Exits and the peculiar nature of Google Street View as a continuous and navigable visualization of actual space, the article draws on multimodal and cognitive narrative theories to analyse the ways in which the geovisual and the verbal interanimate each other. It focuses in particular on immersion as an embodied, sensorial experience, which Larsen enhances by allowing the reader to explore the space of the storyworld from the internal perceptual perspective of the narrator–protagonist. Simultaneously, he self-reflexively discloses fissures and glitches in Google Street View and thus elicits a reflection on the representational capacity of digital geovisuality. The article concludes by suggesting that Entrances & Exits belongs to the vanguard of a new genre of digital imagetexts that integrate textuality, visuality, and haptics.

A Methodology for Efficient Sketch Based Image Retrievals Based on Correlation Based Matching and Generalized Gamma Mixture Model (GGMM)

The huge developments in touch screen devices and used in Computer vision developed the methodologies for recovering the images based on content. However in certain situations narration is the best suitable way to express and hence in the views of narration sketch based images are thus developed and utilized. These sketch based images are most useful in identifying face in criminal investigations. This paper provides a methodology for retrieving such sketch based images using the correlation based matching and generalized gamma mixture models. Performance is measured using precision and recall.

DETERMINATION OF THE BASIC PARAMETERS OF SENSOR DEVICES FOR THE IMPLEMENTATION OF PSYCHONEUROLOGICAL RESEARCH WITH THE INTRODUCTION OF MULTITOUCH TECHNOLOGY

The subject of the article is the testing process on multi-touch devices for the implementation of psychoneurological studies. The goal of the work is development specialized dynamic tests with the introduction of Multi-touch technology to detect carpal tunnel syndrome and hand dysfunction in the early stages of the disease. An objective of the work is to conduct a theoretical analysis to determine the optimal parameters of touchscreen devices for the implementation of psycho neurological testing; based on experimental studies to determine the main physical indicators of the test results, indicating the presence of functional disorders of the hand. The constant use of computers, smartphones, gadgets provide multiple identical load on the wrist; constant visual and neuro-emotional stress; sitting position of the body; static tension of the musculoskeletal system caused by the need to maintain a working posture; decreased overall motor activity. The methods of computer modeling and visualization of dynamic tests using the multi-platform programming language Python were used. Results. The scientific novelty and practical significance of the obtained results is the development of specialized software and algorithmic software for dynamic testing on sensory devices with the ability to video record the movements of the upper extremities in the mode "Rapid". The advantage of this method is its non-invasiveness, ease of implementation, as well as the absence of sensors that can restrict movement. Using the rapid method, one can easily distinguish the pathological state of motor activity of the hand from the established norm. Conclusions. A specialized software tool was developed, which was adapted for multi-touch screens that allow identifying functional disorders of the hand by registering the main physical parameters during testing. Dynamic tests are adapted for any age category of participants (patients), have additional modes for individual color perception and different levels of complexity.

Turns vs consensus: Learning mathematics in multi-touch surfaces

• There is no significant difference between turns and consensus. • Consensus methodology enhaces 21st century skills. • There is no gender gap. • Multi-touch table have a positive impact in the knowledge. The adoption of information and communication technologies in primary education is crucial for adapting traditional classrooms to the digital era. Over time, young children are increasingly using touch screen technologies such as tablets at home and interactive whiteboards at school, either for leisure or for academic activities. However, the literature shows that there is still a gap between what is known about the benefits of using this technology and its real use in primary education settings. Most researchers have focused on the pedagogical theory behind using touch screen devices, but there are few empirical studies about how these technologies and different approaches affect students’ learning processes. This paper presents two learning experiences in a primary school in Fuenlabrada (Madrid-Spain) where primary students performed mathematics activities using a multi-touch table with two different methodologies: turns and consensus. The results show that both methodologies help students acquire meaningful learning, but there is no statistically significance between them.

Making block-based programming accessible to people with cognitive disabilities

Block-based programming languages are a popular way to introduce programming concepts to novice learners. However, block-based languages have inaccessible representations, which can create cognitive difficulties in manipulating blocks, assembling programs, and interpreting code. In my dissertation, I plan to address these problems by developing techniques to make programming accessible using a touch screen device. I will evaluate my designs through usability studies with adults with cognitive disabilities, special educators, and caregivers. My research aims to provide empirical findings on accessible representations and alternative interaction techniques in block-based languages.