Real-time Capture Research Articles

Materialist epistemology lends design wings: educational design as an embodied process

Materialist design is presented as an embodied perspective on educational design that can be applied to redesign of classroom-based learning environments. Materialist design is informed by a framework of materialist epistemology, which positions material innovation on equal placement with symbol-based formal theory. Historical examples of Einstein’s conceptual reliance on trains for his Theory of Relativity and the Wright brothers’ use of wind tunnels in aeronautics illustrate how materialist design drives progress on complex design problems. A key aspect is the application of scale-down methodology, where complex systems are reconceptualized as interactions among nearly decomposable subsystems that can be redesigned and integrated back into the entire system. The application of materialist design is illustrated with the redesign of an embodied video game that uses real-time motion capture technology to promote high school geometry reasoning and proof, following its use in an ethnically and linguistically diverse classroom. Our embodied perspective offers particular insights for understanding and implementing designs of complex learning environments, and assessing their influences on educational practices and student outcomes.

Linearly augmented real-time 4D expressional face capture

Personalised 3D face creation has always been a hot topic in the computer vision community. Many methods have been proposed including the statistic model, the non-rigid registration and high-end depth acquisition equipment. However, in practical applications, those existing methods still have their own limitations. For example, the performance of the statistic model-based methods highly depends on the generality of the pre-trained statistic model; the non-rigid registration based methods are sensitive to the quality of input data; the high-end equipment-based methods are less able to be popularised due to the expensive equipment costs; the deep learning-based methods can only perform well if proper training data provided for the target domain, and require GPU for better performance. To this end, this paper presents an adaptive template augmented method that can automatically obtain a personalised 4D facial modelling only using a consumer-grade device. The noisy data from such a cheap device are well handled. The whole process consists of a series of linear solutions and can be achieved in real-time for online processing only based on the CPU computation on a laptop. There is no constraint nor complex operation required by the proposed method. No additional time-consumptive pre- or post-processing for the personalisation is needed. Comparisons against several existing methods demonstrate the superiority of the proposed method.

Fast and deep facial deformations

Film-quality characters typically display highly complex and expressive facial deformation. The underlying rigs used to animate the deformations of a character's face are often computationally expensive, requiring high-end hardware to deform the mesh at interactive rates. In this paper, we present a method using convolutional neural networks for approximating the mesh deformations of characters' faces. For the models we tested, our approximation runs up to 17 times faster than the original facial rig while still maintaining a high level of fidelity to the original rig. We also propose an extension to the approximation for handling high-frequency deformations such as fine skin wrinkles. While the implementation of the original animation rig depends on an extensive set of proprietary libraries making it difficult to install outside of an in-house development environment, our fast approximation relies on the widely available and easily deployed TensorFlow libraries. In addition to allowing high frame rate evaluation on modest hardware and in a wide range of computing environments, the large speed increase also enables interactive inverse kinematics on the animation rig. We demonstrate our approach and its applicability through interactive character posing and real-time facial performance capture.

A feasibility study of mHealth and wearable technology in late onset GM2 gangliosidosis (Tay-Sachs and Sandhoff Disease)

BackgroundAs part of a late onset GM2 gangliosidosis natural history study, digital health technology was utilized to monitor a group of patients remotely between hospital visits. This approach was explored as a means of capturing continuous data and moving away from focusing only on episodic data captured in traditional study designs. A strong emphasis was placed on real-time capture of symptoms and mobile Patient Reported Outcomes (mPROs) to identify the disease impact important to the patients themselves; an impact that may not always correlate with the measured clinical outcomes assessed during patient visits. This was supported by passive, continuous data capture from a wearable device.ResultsAdherence rate for wearing the device and completing the mPROs was 84 and 91%, respectively, resulting in a rich multidimensional dataset. As expected for a six-month proof-of-concept study in a disease that progresses slowly, statistically significant changes were not expected or observed in the clinical, mPROs, or wearable device data.ConclusionsThe study demonstrated that patients were very enthusiastic and motivated to engage with the technology as demonstrated by excellent compliance. The combination of mPROs and wearables generates feature-rich datasets that could be a useful and feasible way to capture remote, real-time insight into disease burden.

Performance Analysis of Video On-demand and Live Video Streaming using Cloud based Services

The advent of Cyber-Physical Systems (CPS) has brought a revolutionary change coined as a mixture of information, communication, computation, and control. With applications in smart grid, health monitoring, automatic avionics, distributed robotics, etc., CPS is currently an area of attention among the academia and industry. The advancement of mobile communications and embedded technology has made it possible to build large scale CPS consisting of the interconnection of mobile phones. These devices collect information about the surrounding environment at any time anywhere basis through real-time video capture. Video streaming has proven to be a massive industry that is growing rapidly playing an important role in everyday life. Customer-driven approach wanting best experience with quality has to be the core offering of contemporary scenario. Video streaming is categorized into Video-On-Demand Streaming (VoDS) and Live Video Streaming (LVS) showing the current state-of-art opportunities. Many diverse applications of video streaming are military video surveillance using drones, live sports match player face recognition, on-demand video characters recognition, movie summarization like identifying parts of the movie which are viewed many times by different users, movie and series recognition, motion detection, gesture recognition, image segmentation, etc. This paper introduces an approach to develop video analysis on VoD and LVS using cloud-based services and analyzes the impact of Quality of Experience (QoE), cost, and bandwidth on the cloud. To achieve the best user experience for video streaming and video analysis, Content Delivery Network (CDN) offers the best QoE at various analyzed locations using various cloud providers like Amazon Web Services (AWS) CloudFront, Google Cloud CDN, Azure CDN, Akamai CDN, etc.

Design and Modeling of Resource-constrained IoT based Body Area Networks

Due to the recent advancement and development of sensing, wireless, and communication technologies, therehas been a shift in attention towards Body Area Networks (BANs). One of the most important services of BAN is the remote monitoring of patients, enabling doctors to observe, diagnose, and prescribe the patients without being physically present. Various vital signs are being monitored by body sensing devices installed inside, on or off the body of patients, but most of these devices are constrained in terms of resources such as storage, processing, bandwidth, and energy due to their smaller size. This paper aims at highlighting the key findings related to BAN applications, constrained resources, and various resource management techniques. The paper also presents the design and modeling of a resource-constrained BAN system and discusses the various scenarios of BAN in the context of resource constraints. It further proposes an Advanced Edge Clustering (AEC) approach to manage the resources such as energy, storage, and processing of BAN devices while performing real-time data capture of critical health parameters and detection of abnormal patterns. The comparison of the AEC approach is done with the Stable Election Protocol (SEP) through simulations and empirical data analysis. The results show an improvement in energy, processing time and storage requirements for the processing of data on BAN devices in AEC as compared to SEP.

Introduction to the Special Issue on Evolving IoT and Cyber-Physical Systems: Advancements, Applications, and Solutions

Introduction to the Special Issue on Evolving IoT and Cyber-Physical Systems: Advancements, Applications, and Solutions

Assessment of VaxTrac electronic immunization registry in an urban district in Sierra Leone: Implications for data quality, defaulter tracking, and policy

BackgroundIn 2016, the Sierra Leone Ministry of Health and Sanitation (MoHS) piloted VaxTrac, an electronic immunization registry (EIR), in an urban district to improve management of vaccination records and tracking of children who missed scheduled doses. We aimed to document lessons learned to inform decision-making on VaxTrac and similar EIRs’ future use. MethodsTen out of 50 urban health facilities that implemented VaxTrac were purposively selected for inclusion in a rapid mixed-method assessment from November to December 2017. For a one-month period, records of six scheduled vaccine doses among children < 2 years old in VaxTrac were abstracted and compared to three paper-based records (register of under-two children, daily tally sheet, and monthly summary form). We used the under-two register as the reference gold standard for comparison purposes. We interviewed and observed 10 heath workers, one from each selected facility, who were using VaxTrac. ResultsOverall, VaxTrac captured < 65% of the vaccine doses reported in the paper-based sources, but in the largest health facility VaxTrac captured the highest number of doses. Two additional notable patterns emerged: 1) the aggregated data sources reported higher doses administered compared to the under-two register and VaxTrac; 2) data sources that need real-time data capture during the vaccination session reported fewer doses administered compared to the monthly HF2 summary form. Health workers expressed that the EIR helped them to shorten the time to manage, summarize, and report vaccination records. Workflows for data entry in VaxTrac were inconsistent among facilities and rarely integrated into existing processes. Data sharing restrictions contributed to duplicate records. ConclusionAlthough VaxTrac helped to shorten the time to manage, summarize, and report vaccination records, data sharing restrictions coupled with inconsistent and inefficient workflows were major implementation challenges. Readiness-to-introduce and sustainability should be carefully considered before implementing an EIR.

The Influence of Immersive Virtual Reality Systems on Online Social Application

This study presents a face-to-face simulation of social interaction based on scene roaming, real-time voice capture, and action capture. This paper aimed to compare the difference between social and traditional plane social communication, analyzing its advantages and shortcomings. In particular, we designed an experiment to study the performance of face-to-face simulation based on virtual reality technology, observing the adaptability of the user to the system and the accuracy of body language recognition. We developed an immersive virtual reality social application (IVRSA), which has a basic social function. As an experimental platform, IVRSA uses Unity3D as its engine, and HTC VIVE as an external input/output device, with voice communication, immersive virtual tour, and head and hand movement simulation functions. We recruited 30 volunteers for the test. The test consisted of two parts. The first part was to provide a news topic for volunteers to freely communicate in IVRSA and WeChat. After communication, we used questionnaires to obtain feedback results of the test and compared the two social applications. In the second part, some volunteers were given a list of actions, which they were asked to describe to the rest of volunteers in the form of body expression, letting them guess the action they were performing. After the end of test, the accuracy rate and the time used were analyzed. Results showed that user’s intention expression efficiency was very high in the immersive virtual reality social system. Body tracking devices with more tracking points can provide better body expression effect.

BIM-based performance monitoring for smart building management

PurposeEffective use of post-occupancy evaluation (POE) data – quantitative physical measurements and qualitative occupants’ perceptions are limited due to practical challenges and research gaps. Although building information modelling (BIM) has enabled a paradigm shift in the architecture, engineering and construction (AEC) industries, its use in facility management (FM) is still infancy. Limited research has used building performance data to enable changes to BIM models for the benefit of FM. This paper aims to propose the innovative process to collect and contextualize two fragmented types of POE data sets by filling methodological gap in POE research. Moreover, it presents innovative modelling techniques to facilitate BIM as a more effective platform to visualize such currently fragmented data sets in real-time while enabling a decision-making model to benefit facility managers.Design/methodology/approachThe paper presents a process of capturing cloud-based POE data, both wireless sensor network-based physical measurement data and mobile app-based occupant perception data. Real-time capture and visualization of such building performance data was demonstrated through a pilot data collection. Subsequently, the innovative visualization of the cloud connected data is supported by a prototype game engine-based BIM model.FindingsCloud-based POE data, both quantitative physical measurements and qualitative occupants’ reported perceptions, can be effectively used in FM practice with the use of innovative data capture and visualization techniques in a beneficial manner for facility operation and management decisions. This paper also demonstrates the ability of BIM to serve as a “single source of truth” to support post-construction building performance data.Originality/valueWhile addressing a number of research gaps, the paper provides a holistic approach to BIM-based performance monitoring for smart FM to achieve the ultimate vision of BIM enabled FM. The innovative system is expected to provide a powerful and practical tool for data collection, analysis and visualization for intelligent facility management decision making. This paper contributes to fill an important research and practice gap in the area of next generation smart building management practices.

Super Sub-Nyquist Single-Pixel Imaging by Total Variation Ascending Ordering of the Hadamard Basis

Single pixel imaging (SPI) captures images without array detectors or raster scanning. When combined with compressive sensing techniques it enables novel solutions for high-speed optical imaging and spectroscopy. However, when it comes to the real-time capture and analysis of a fast event, the challenge is the inherent trade-off between frame rate and image resolution. Due to the lack of sufficient sparsity and the intrinsic iterative process, conventional compressed sensing techniques have limited improvement in capturing natural scenes and displaying the images in real time. In this work, we demonstrate a novel alternative compressive imaging approach employing an efficient and easy-implementation sampling scheme based on reordering the deterministic Hadamard basis through their total variation. By this means, the number of measurements and acquisition are reduced significantly without needing complex minimization algorithms. We can recover a 128 × 128 image with a sampling ratio of 5% at the signal peak signal-to-noise ratio (PSNR) of 23.8 dB, achieving super sub-Nyquist sampling SPI. Compared to other widely used sampling e.g. standard Hadamard protocols and Gaussian matrix methods, this approach results in a significant improvement both in the compression ratio and image reconstruction quality, enabling SPI for high frame rate imaging or video applications.

Digital‐twin‐based implementation framework of production service system for highly dynamic production logistics operation

In the customised production mode, production and logistics are required to operate in synchronisation under the high dynamic interference, to realise the rapid response to the personalised demand of high-frequency changes. Facing high dynamic interference, how to realise the efficient synchronised operation of production and logistics by integrating external service resources has become a major challenge for manufacturing manufacturers. This study proposes a digital-twin-based implementation framework of production service system (DTIF-PnSS) for highly dynamic production logistics operations. The framework realises online collaborative operation and autonomous decision-making control of production logistics under high dynamic interference by providing smart services for real-time dynamic capture, operational precision mapping, dynamic virtual simulation and iterative decision optimisation. At last, the effectiveness of the DTIF-PnSS is verified by an application case of a coating chemical enterprise. It provides a feasible way to support the production logistics synchronised operation under high dynamic interference.

Precision Nutrition Impact on Health-Related Behavior Change in Service Members

Abstract Objectives 1) Examine the effectiveness of gene-based nutrition counseling on health-related behaviors in service members as measured by body weight, body mass index, blood glucose, lipids, 25-hydroxyvitamin D, % body fat, waist circumference, and blood pressure; 2) Evaluate the feasibility of a digital application to accurately capture diet, activity, and sleep behaviors; 3) Describe military-unique characteristics in demographics, diet, and lifestyle for Northwest Army and Southwest Air Force cohorts Methods In this prospective RCT, a baseline genomic profile built from 70 genes/80 variants following amplicon sequencing on a MiSeq platform is augmented by serum biomarkers specific to diet-related chronic disease risk (metabolic syndrome and vitamin D deficiency) for each subject. Treatment group receives gene-based nutrition counseling; control group receives evidence-based nutrition pamphlets. A digital app is used for real-time health data capture, validated by in-person RD interviews. After the initial 6 weeks, both groups use study resources independently and return for final body composition, BP, and serum biomarkers at 12 weeks. Results Preliminary findings: Northwest cohort has enrolled 26 of 75 active duty soldiers. Sample has slightly more males (54%), mean age 29.6 years, 64% married, 77% Caucasian, non-Hispanic, and 60% report some college. About 50% lost an average of 15 lbs to join the Army. Mean anthropometrics for females: BMI 27.4, 30.3% body fat, waist circ 32.2 in; for males: BMI 30.7, 28% body fat, waist circ 40 in. High variant presence noted for obesity and hypertension. Signals of abnormal biomarkers are insignificant although 60% of sample has baseline serum vitamin D less than 30 ng/mL. Conclusions This initiative has begun to advance the precision nutrition agenda in the military setting where advanced practice RNs and RDs will soon use genomic risk data to guide diet and lifestyle counseling to support personal and military readiness and optimize mission performance. Funding Sources TriService Nursing Research Program.

Exploration of the Application of Virtual Reality and Internet of Things in Film and Television Production Mode

In order to reduce some of the problems of technological restructuring and insufficient expansion in the current film and television production mode, the application of emerging technologies such as artificial intelligence (AI), virtual reality (VR), and Internet of Things (IoT) in the film and television industry is introduced in this research. First, a topical crawler tool was constructed to grab relevant texts about “AI”, “VR”, and “IoT” crossover “film and television”, and the grasping accuracy rate and recall rate of this tool were compared. Then, based on the extracted text, the data of recent development in related fields were extracted. The AdaBoost algorithm was used to improve the BP (Back Propagation) neural network (BPNN). This model was used to predict the future development scale of related fields. Finally, a virtual character interaction system based on IoT-sensor technology was built and its performance was tested. The results showed that the topical crawler tool constructed in this study had higher recall rate and accuracy than other tools, and a total of 188 texts related to AI, VR, and IoT crossover television films were selected after Naive Bayes classification. In addition, the error of the BPNN prediction model based on the AdaBoost algorithm was less than 20%, and it can effectively predict the future development scale of AI and other fields. In addition, the virtual character interaction system based on IoT technology constructed in this study has a high motion recognition rate, produces a strong sense of immersion among users, and can realize real-time capture and imitation of character movements. In a word, the field of AI and VR crossover film and television has great development prospects in the future. Therefore, the application of IoT technology in building the virtual-character interaction system can improve the effect of VR or AI film and television production.

Object Detection and Recognition for Visually Impaired People

Good vision is an expensive gift but now a day’s loss of vision is becoming common issue. Blind or visually impaired people does not have any conscious about the danger they are facing in their daily life. To help the blind people the visual world has to be transformed into the audio world with the potential to inform them about objects. Various challenges are faced by visually impaired patients even in the familiar environment. Visually impaired individuals are at drawback due to lack of sufficient information about their familiar environment. This project employs a Convolution Neural Network for recognition of pre-trained objects. This project employs in deep learning a deep Neural Network (DNN) for recognizing the object which is captured from the real world. The captured image is compared with some pre trained objects that is stored in dataset .The comparative of the object is based on the shape and size of an objects. In deep neural network, TensorFlow package using a model called Mobile Net SSD that is comparing the real time capture image with pre trained object based on shape, size of the object. If the image is matched with that trained object, it will display the name of the object. Then the name of the object is converted into audio output with the help of gTTS. This will helps to identify and detect what object is present in front of blind people and give output as audio.

A Road Monitoring Approach with Real-Time Capturing of Events for Efficient Vehicles Safety in Smart City

Nowadays, Internet-of-things (IoT) provides various services to the users by equipped with the smart devices. In this regard, the next generation of vehicles collaborates with the features of IoT to provide the safety and security on the roads. To achieve this, it equipped with short-range communication advances and establishes Vehicle-to-Vehicle (V2V) connectivity. The standardized V2V connectivity and communication are termed as IEEE 802.11p. Later, an alternative named as (LTE-V2V) has been introduced. However, both the technologies are only concerned with the continuous broadcast of information and cooperative awareness. It only takes information from one vehicle in the format of text message and sends to another. In this regard, an efficient and satisfactory safety is not provided by these technologies for analysis of real-time road traffic monitoring. A clear observation of road conditions and awareness of event triggering is a necessary task rather than broadcasting of information provided by the vehicles. Therefore in this paper, we proposed a solution by providing the real-time information about road conditions and traffic scenarios to the users. We utilized the capturing images of road conditions by the positioned cameras and Global Positioning System in order to extract the information regarding vehicle and camera position. The proposed work provides a better security rather than message passing system in V2V communication. The users in our anticipated scenarios can extract and see a clear view of road conditions by use of captured videos/images. We offered an approach named as Smart Road Monitoring for IoT based smart cities to provide an effective solution for the real-time analysis of traffic road conditions. The proposed approach gives a clear view of conditions of the road to the users such that they can make a decision according to their suitability Our proposed solution well copes with the moderate traffic conditions and provides high satisfaction score. The performance analysis is done in a comparison of V2V and LTE-V2V in terms of various metrics. The simulation results show that our proposed work can achieve high performance in the provision of providing safety compared to other scheme introduced in this field.

Passthrough+

We present an end-to-end system for real-time environment capture, 3D reconstruction, and stereoscopic view synthesis on a mobile VR headset. Our solution allows the user to use the cameras on their VR headset as their eyes to see and interact with the real world while still wearing their headset, a feature often referred to as Passthrough. The central challenge when building such a system is the choice and implementation of algorithms under the strict compute, power, and performance constraints imposed by the target user experience and mobile platform. A key contribution of this paper is a complete description of a corresponding system that performs temporally stable passthrough rendering at 72 Hz with only 200 mW power consumption on a mobile Snapdragon 835 platform. Our algorithmic contributions for enabling this performance include the computation of a coarse 3D scene proxy on the embedded video encoding hardware, followed by a depth densification and filtering step, and finally stereoscopic texturing and spatio-temporal up-sampling. We provide a detailed discussion and evaluation of the challenges we encountered, as well as algorithm and performance trade-offs in terms of compute and resulting passthrough quality.;AB@The described system is available to users as the Passthrough+ feature on Oculus Quest. We believe that by publishing the underlying system and methods, we provide valuable insights to the community on how to design and implement real-time environment sensing and rendering on heavily resource constrained hardware.

Utilizing the F3App to Capture Learner Feedback About Faculty Teaching.

The Accreditation Council for Graduate Medical Education requires soliciting learner feedback on faculty teaching, although gathering meaningful feedback is challenging in the medical education environment. We developed the Faculty Feedback Facilitator (F3App), a mobile application that allows for real-time capture of narrative feedback by residents. The purpose of our study was to assess efficacy, usability, and acceptability of the F3App in family medicine residency programs. Residents, faculty, and program directors (PDs) from eight residency programs participated in a beta test of the F3App from November 2017 to May 2018; participants completed pre- and postimplementation surveys about their evaluation process and the F3App. We interviewed PDs, and analyzed responses using a thematic analysis approach. Survey results showed significant postimplementation increases in faculty agreement that accessing evaluations is easy (42%), evaluations are an effective way to communicate feedback (34%), feedback is actionable and meaningful (24%), and the current system provides meaningful data for promotion (33%). Among residents, agreement that the current system allows meaningful information sharing and is easy to use increased significantly, by 17% each. The proportion of residents agreeing they were comfortable providing constructive criticism increased significantly (22%). PDs generally reported that residents were receptive to using the F3App, found it quick and easy to use, and that feedback provided was meaningful. Participating programs reviewed the F3App positively as a tool to gather narrative feedback from learners on faculty teaching.

Methods and design for the ADAPT study: Application of integrateD Approaches to understanding Physical activity during the Transition to emerging adulthood

BackgroundThe overarching objective of the study is to obtain a comprehensive understanding of the salient factors predicting changes in physical activity (PA) during adolescents’ transition into emerging adulthood. Using the Multi-Process Action Control model as our guiding framework, we will examine how implicit and explicit psychological processes along with regulatory practices impact PA change during this major life transition. Additionally, we will use a real-time data capture method called Ecological Momentary Assessment to further investigate how environmental and contextual factors, and momentary psychosocial influences effect PA patterns across this dynamic life stage.MethodsThe ADAPT study is a 4-year project comprised of two interrelated studies. Study I is a large prospective cohort study that will invite all grade 11 students across one large school board (a total of seven secondary schools) to participate by completing an online questionnaire. Using a cluster randomization approach, a subset of students from each school will be invited to participate in Study II, whereby participants will wear an accelerometer and complete Ecological Momentary Assessments 5 times a day over a 7-day study period. For both studies, following baseline assessments, there will be three annual follow-up assessments approximately 12 months apart.DiscussionThe current study represents one of the largest longitudinal cohort studies examining PA and its determinants and associated consequences among adolescents transitioning out of high school into emerging adulthood. Findings from this study will provide a much more in-depth understanding of how and why changes in PA behaviour occur across this first major life transition.

Real-Time Optimal Approach and Capture of ENVISAT Based on Neural Networks

A neural network-based controller is developed to enable a chaser spacecraft to approach and capture a disabled Environmental Satellite (ENVISAT). This task is conventionally tackled by framing it as an optimal control problem. However, the optimization of such a problem is computationally expensive and not suitable for onboard implementation. In this work, a learning-based approach is used to rapidly generate the control outputs of the controller based on a series of training samples. These training samples are generated by solving multiple optimal control problems with successive iterations. Then, Radial Basis Function (RBF) neural networks are designed to mimic this optimal control strategy from the generated data. Compared with a traditional controller, the neural network controller is able to generate real-time high-quality control policies by simply passing the input through the feedforward neural network.