Conventional Video Research Articles

MirrorCampus: A Synchronous Hybrid Learning Environment That Supports Spatial Localization of Learners for Facilitating Discussion-Oriented Behaviors

A growing number of higher-education institutions are implementing synchronous hybrid delivery, which provides both online and on-campus learners with simultaneous instruction, especially for facilitating discussions in Active Learning (AL) contexts. However, learners face difficulties in picking up social cues and gaining free access to speaking rights due to the geometrical misalignment of individuals mediated through screens. We assume that the cultivation of discussions is allowed by ensuring the spatial localization of learners similar to that in a physical space. This study aims to design a synchronous hybrid learning environment, called Mirror Campus (MC), suitable for the AL scenario that connects physical and cyberspaces by providing spatial localization of learners. We hypothesize that the MC promotes discussion-oriented behaviors, and eventually enhances applied skills for group tasks, related to discussion, creativity, decision-making, and interdependence. We conducted an experiment with five different groups, where four participants in each group were asked to discuss a given topic for fifteen minutes, and clarified that the occurrences of facing behaviors, intervening, and simultaneous utterances in the MC were significantly increased compared to a conventional video conferencing. In conclusion, this study demonstrated the significance of the spatial localization of learners to facilitate discussion-oriented behaviors such as facing and speech.

Deep Learning-Based Watermarking Techniques Challenges: A Review of Current and Future Trends

The digital revolution places great emphasis on digital media watermarking due to the increased vulnerability of multimedia content to unauthorized alterations. Recently, in the digital boom in the technology of hiding data, research has been tending to perform watermarking with numerous architectures of deep learning, which has explored a variety of problems since its inception. Several watermarking approaches based on deep learning have been proposed, and they have proven their efficiency compared to traditional methods. This paper summarizes recent developments in conventional and deep learning image and video watermarking techniques. It shows that although there are many conventional techniques focused on video watermarking, there are yet to be any deep learning models focusing on this area; however, for image watermarking, different deep learning-based techniques where efficiency in invisibility and robustness depends on the used network architecture are observed. This study has been concluded by discussing possible research directions in deep learning-based video watermarking.

An investigation of machine learning-based video compression techniques

As video technology continues to seamlessly weave itself into the fabric of daily life, there is a growing need for enhanced storage and efficient video transmission. This surge in demand has led to heightened expectations and standards for video compression technology. Machine learning as an up-and-coming technology can play its advantages in the field of video compression. This article reviews the current state of research on combining video compression techniques with machine learning. The article provides an overview of various research avenues for enhancement, spanning from conventional video compression algorithms to the fusion of traditional compression frameworks with machine learning methodologies, and even the development of novel end-to-end compression algorithms. In additional, the article explores the possible various application scenarios of machine learning-based video compression algorithms based on the characteristics of such non-standard and arithmetic demanding algorithms. At the end, the article speculates on the future of video compression algorithms based on the content of the various studies reviewed in the article.

Exploring the impact of immersive virtual reality on depression knowledge and stigma reduction: a cross-over RCT fMRI study

The stigma of mental illness is a form of negative judgmental knowledge and is a barrier to individual seeking treatment. Contact-based educational interventions with first-person perspective (1PP) combined with immersive virtual reality (IVR) is promising as an anti-stigma intervention. This study aims to investigate the effectiveness of 1PP anti-stigma IVR intervention compared to video in enhancing depression knowledge and reducing stigma, as well as to examine the corresponding depression knowledge brain activity change using functional magnetic resonance imaging (fMRI). Participants engaged in a 1PP anti-stigma intervention using both IVR and conventional video, focusing on the daily life and recovery of a patient with mild depression. The change in depression knowledge, stigma-related behavioral, and brain activity using fMRI were measured at pre- and post-interventions. Depression knowledge improved for both interventions; however, only the IVR intervention reduced stigma. In the IVR intervention, depression knowledge score was positively associated with neural response in the right superior frontal gyrus activation, indicative of empathic concern. Conversely, the video intervention correlated with increased activity in the right anterior insula, suggesting a distress-related response. The findings demonstrate that the immersive nature of IVR can reduce stigma more effectively than video intervention. This effectiveness is underpinned by the change in depression knowledge on neural activity, with IVR fostering empathy-related behavioral responses. The results highlight the potential of IVR in enhancing empathic understanding and reducing stigma towards mental illness, emphasizing the need for further exploration into immersive technologies for mental health education.

Effectiveness of Peer-Based and Conventional Video Education in Reducing Perioperative Depression and Anxiety Among Coronary Artery Bypass Grafting Patients: A Randomized Controlled Trial

PurposeDepression and anxiety are common comorbidities in patients undergoing coronary artery bypass grafting (CABG), with potential adverse effects on surgical outcomes. Effective interventions to reduce depression and anxiety in these patients are therefore warranted. This study investigated the effectiveness of peer-based video education compared to conventional video education in reducing perioperative depression and anxiety in CABG patients. DesignA three-arm, parallel, randomized, controlled trial design was employed. MethodsA total of 114 participants were randomly assigned to one of three groups (n=38 per group): standard education (control), conventional video education, and peer-based video education. State anxiety levels were measured using the Spielberger State Anxiety Questionnaire at one day before surgery (baseline), one hour before surgery, and four weeks after surgery. Depression levels were measured using the Beck Depression Inventory Short-Form (BDI-SF) at baseline and four weeks after surgery. Statistical analyses, including Chi-square, Fisher's exact test, one-way analysis of variance (ANOVA), and repeated measures ANOVA, were applied to analyze the collected data. FindingsBoth peer-based and conventional video education groups demonstrated lower preoperative anxiety levels compared to the control group. However, only the peer-based video education group exhibited a statistically significant difference (p < 0.05). Four weeks after surgery, anxiety and depression levels decreased in all participants compared to baseline, with no statistically significant differences among the three groups. ConclusionOur findings suggest that peer-based video education is more effective in controlling preoperative anxiety in patients undergoing CABG than conventional video education and standard education. Moreover, video-based education, whether conventional or peer-based, appears to be as effective as standard education in reducing anxiety and depression four weeks after CABG surgery. Further research is warranted to investigate the influence of content and presentation methods on patient outcomes and explore the potential long-term benefits of video-based education in promoting patient care.

Lossless Encoding of Time-Aggregated Neuromorphic Vision Sensor Data Based on Point-Cloud Compression.

Neuromorphic Vision Sensors (NVSs) are emerging sensors that acquire visual information asynchronously when changes occur in the scene. Their advantages versus synchronous capturing (frame-based video) include a low power consumption, a high dynamic range, an extremely high temporal resolution, and lower data rates. Although the acquisition strategy already results in much lower data rates than conventional video, NVS data can be further compressed. For this purpose, we recently proposed Time Aggregation-based Lossless Video Encoding for Neuromorphic Vision Sensor Data (TALVEN), consisting in the time aggregation of NVS events in the form of pixel-based event histograms, arrangement of the data in a specific format, and lossless compression inspired by video encoding. In this paper, we still leverage time aggregation but, rather than performing encoding inspired by frame-based video coding, we encode an appropriate representation of the time-aggregated data via point-cloud compression (similar to another one of our previous works, where time aggregation was not used). The proposed strategy, Time-Aggregated Lossless Encoding of Events based on Point-Cloud Compression (TALEN-PCC), outperforms the originally proposed TALVEN encoding strategy for the content in the considered dataset. The gain in terms of the compression ratio is the highest for low-event rate and low-complexity scenes, whereas the improvement is minimal for high-complexity and high-event rate scenes. According to experiments on outdoor and indoor spike event data, TALEN-PCC achieves higher compression gains for time aggregation intervals of more than 5 ms. However, the compression gains are lower when compared to state-of-the-art approaches for time aggregation intervals of less than 5 ms.

A Survey on Video Streaming for Next-Generation Vehicular Networks

As assisted driving technology advances and vehicle entertainment systems rapidly develop, future vehicles will become mobile cinemas, where passengers can use various multimedia applications in the car. In recent years, the progress in multimedia technology has given rise to immersive video experiences. In addition to conventional 2D videos, 360° videos are gaining popularity, and volumetric videos, which can offer users a better immersive experience, have been discussed. However, these applications place high demands on network capabilities, leading to a dependence on next-generation wireless communication technology to address network bottlenecks. Therefore, this study provides an exhaustive overview of the latest advancements in video streaming over vehicular networks. First, we introduce related work and background knowledge, and provide an overview of recent developments in vehicular networking and video types. Next, we detail various video processing technologies, including the latest released standards. Detailed explanations are provided for network strategies and wireless communication technologies that can optimize video transmission in vehicular networks, paying special attention to the relevant literature regarding the current development of 6G technology that is applied to vehicle communication. Finally, we proposed future research directions and challenges. Building upon the technologies introduced in this paper and considering diverse applications, we suggest a suitable vehicular network architecture for next-generation video transmission.

Conventional video recordings dependably quantify whole-body lifting strategy using the Stoop-Squat-Index: A methods comparison against motion capture and a reliability study

Whole-body lifting strategies could be derived from conventional video recordings using the Stoop-Squat-Index, which quantifies the ratio between trunk forward lean and lower extremity joint flexion from 0 (full squat) to 100 (full stoop). The purpose of this study was to compare Stoop-Squat-Indices derived from conventional video recordings to those from a three-dimensional marker-based motion capture system and to evaluate interrater and intrarater reliability of the video-based approach.Thirty healthy participants lifted a 5-kg box under different conditions (freestyle, squat, stoop). Kinematic data were recorded using a Vicon motion capture system (serving as reference standard) and an iPad camera. Stoop-Squat-Indices over the entire lifting cycle were derived separately from both approaches. Agreement was assessed using mean differences (video minus motion capture) and limits of agreement. Reliability was investigated by calculating intraclass correlation coefficients (ICC) and minimal detectable changes (MDC) over the course of the lifting cycle. Systematic errors were identified with Statistical Parametric Mapping-based T-tests.Systematic errors between the video-based and the motion capture-based approach were observed among all conditions. Mean differences in Stoop-Squat-Indices over the lifting cycle ranged from −6.9 to 3.2 (freestyle), from −1.8 to 5.3 (squat) and from −2.8 to −1.1 (stoop). Limits of agreement were lower when the box was close to the floor, and higher towards upright standing. Reliability of the video-based approach was excellent for most of the lifting cycle, with ICC above 0.995 and MDC below 3.5.These findings support using a video-based assessment of Stoop-Squat-Indices to quantify whole-body lifting strategy in field.

Personalized recognition system in online shopping by using deep learning

This study presents an effective monitoring system to watch the Buying Experience across multiple shop interactions based on the refinement of the information derived from physiological data and facial expressions. The system's efficacy in recognizing consumers' emotions and avoiding bias based on age, race, and evaluation gender in a pilot study. The system's data has been compared to the outcomes of conventional video analysis. The study's conclusions indicate that the suggested approach can aid in the analysis of consumer experience in a store setting.

A 360° video as visual training support for independent movement acquisition—benefit evaluation with the TAM

A 360° video combines the potential of conventional video technology with immersive–interactive design opportunities. The multi-perspective viewing option within a 360° video also enables possible applications as visual training support for motor learning. There are already first methodological–didactic approaches to movement learning; nevertheless, research on 360° video technology can largely be classified as exploratory. This article presents the use of 360° video as a visual training aid for the independent movement acquisition of various fascial movement exercises. In an intervention study, two randomized groups (N = 48) were used to evaluate the subjective benefit. Following the technology acceptance model, subjective benefit perception and rating of format were compared after two crossover interventions with 360° videos and/or conventional training videos. No significant differences were found for perceived usefulness (z = −1.014, p = 0.31, r = 0.105) or perceived ease of use (z = −1.278, p = 0.201, r = 0.132). The same applies for intensity of use (z = −0.247, p = 0.805, r = 0.025) and overall subjective rating (z = −1.745, p = 0.081, r = 0.18). Although no significant differences were found in the evaluation of benefits, the participants tended to use 360° videos as visual training support (M = 3.4, SD = 1.581) on a 7-point scale (1 = 360° videos). Although a generalized statement on 360° videos as visual training support is not possible due to low effect strengths, it can be stated that 360° videos are perceived at least as useful as visual training tools.

Event-Enhanced Snapshot Compressive Videography at 10K FPS.

Video snapshot compressive imaging (SCI) encodes the target dynamic scene compactly into a snapshot and reconstructs its high-speed frame sequence afterward, greatly reducing the required data footprint and transmission bandwidth as well as enabling high-speed imaging with a low frame rate intensity camera. In implementation, high-speed dynamics are encoded via temporally varying patterns, and only frames at corresponding temporal intervals can be reconstructed, while the dynamics occurring between consecutive frames are lost. To unlock the potential of conventional snapshot compressive videography, we propose a novel hybrid "intensity + event" imaging scheme by incorporating an event camera into a video SCI setup. Our proposed system consists of a dual-path optical setup to record the coded intensity measurement and intermediate event signals simultaneously, which is compact and photon-efficient by collecting the half photons discarded in conventional video SCI. Correspondingly, we developed a dual-branch Transformer utilizing the reciprocal relationship between two data modes to decode dense video frames. Extensive experiments on both simulated and real-captured data demonstrate our superiority to state-of-the-art video SCI and video frame interpolation (VFI) methods. Benefiting from the new hybrid design leveraging both intrinsic redundancy in videos and the unique feature of event cameras, we achieve high-quality videography at 0.1ms time intervals with a low-cost CMOS image sensor working at 24 FPS.

Omnidirectional Video Super-Resolution Using Deep Learning

Omnidirectional Videos (or 360° videos) are widely used in Virtual Reality (VR) to facilitate immersive and interactive viewing experiences. However, the limited spatial resolution in 360° videos does not allow for each degree of view to be represented with adequate pixels, limiting the visual quality offered in the immersive experience. Deep learning Video Super-Resolution (VSR) techniques used for conventional videos could provide a promising software-based solution; however, these techniques do not tackle the distortion present in equirectangular projections of 360° video signals. An additional obstacle is the limited 360° video datasets to study. To address these issues, this paper creates a novel 360° Video Dataset (360VDS) with a study of the extensibility of conventional VSR models to 360° videos. This paper further proposes a novel deep learning model for 360° Video Super-Resolution (360° VSR), called Spherical Signal Super-resolution with a Proportioned Optimisation (S3PO). S3PO adopts recurrent modelling with an attention mechanism, unbound from conventional VSR techniques like alignment. With a purpose-built feature extractor and a novel loss-function addressing spherical distortion, S3PO outperforms most state-of-the-art conventional VSR models and 360° specific super-resolution models on 360° video datasets. A step-wise ablation study is presented to understand and demonstrate the impact of the chosen architectural sub-components, targeted training and optimisation.

AI-powered real-time annotations during urologic surgery: The future of training and quality metrics

Introduction and objectiveReal-time artificial intelligence (AI) annotation of the surgical field has the potential to automatically extract information from surgical videos, helping to create a robust surgical atlas. This content can be used for surgical education and qualitative initiatives. We demonstrate the first use of AI in urologic robotic surgery to capture live surgical video and annotate key surgical steps and safety milestones in real-time. Summary background dataWhile AI models possess the capability to generate automated annotations based on a collection of video images, the real-time implementation of such technology in urological robotic surgery to aid surgeon and training staff it is still pending to be studied. MethodsWe conducted an educational symposium, which broadcasted 2 live procedures, a robotic-assisted radical prostatectomy (RARP) and a robotic-assisted partial nephrectomy (RAPN). A surgical AI platform system (Theator, Palo Alto, CA) generated real-time annotations and identified operative safety milestones. This was achieved through trained algorithms, conventional video recognition, and novel Video Transfer Network technology which captures clips in full context, enabling automatic recognition and surgical mapping in real-time. ResultsReal-time AI annotations for procedure #1, RARP, are found in Table 1. The safety milestone annotations included the apical safety maneuver and deliberate views of structures such as the external iliac vessels and the obturator nerve. Real-time AI annotations for procedure #2, RAPN, are found in Table 1. Safety milestones included deliberate views of structures such as the gonadal vessels and the ureter. AI annotated surgical events included intraoperative ultrasound, temporary clip application and removal, hemostatic powder application, and notable hemorrhage. ConclusionsFor the first time, surgical intelligence successfully showcased real-time AI annotations of 2 separate urologic robotic procedures during a live telecast. These annotations may provide the technological framework for send automatic notifications to clinical or operational stakeholders. This technology is a first step in real-time intraoperative decision support, leveraging big data to improve the quality of surgical care, potentially improve surgical outcomes, and support training and education.

ComplexEye: a multi-lens array microscope for high-throughput embedded immune cell migration analysis

Autonomous migration is essential for the function of immune cells such as neutrophils and plays an important role in numerous diseases. The ability to routinely measure or target it would offer a wealth of clinical applications. Video microscopy of live cells is ideal for migration analysis, but cannot be performed at sufficiently high-throughput (HT). Here we introduce ComplexEye, an array microscope with 16 independent aberration-corrected glass lenses spaced at the pitch of a 96-well plate to produce high-resolution movies of migrating cells. With the system, we enable HT migration analysis of immune cells in 96- and 384-well plates with very energy-efficient performance. We demonstrate that the system can measure multiple clinical samples simultaneously. Furthermore, we screen 1000 compounds and identify 17 modifiers of migration in human neutrophils in just 4 days, a task that requires 60-times longer with a conventional video microscope. ComplexEye thus opens the field of phenotypic HT migration screens and enables routine migration analysis for the clinical setting.

A Fast Method for Robust Video Watermarking Based on Zernike Moments

Watermarking by Zernike moments has been proven to be effective in providing high rotational resistance. However, due to the high computational complexity, the conventional video watermarking methods using Zernike moments are developed for videos with low resolution. Moreover, according to the properties of Zernike moments, only the matrices of equal height and width can be calculated since the inscribed circle of the original image matrix is selected as the area to be processed, but most of the available videos on the Internet do not meet such requirement. To solve the above problem, this paper proposes a fast watermarking method based on Zernike moments for high resolution videos to resist various attacks. In the proposed method, the frames of a video sequence are firstly grouped, from which a certain number of frame pairs are then selected for watermark embedding. For each frame pair to be embedded, we partition one frame into a set of disjoint blocks and apply singular value decomposition to each block to obtain a square feature matrix. Thereafter, by calculating all the Zernike moments, secret information is embedded into the selected Zernike moments to achieve superior robustness while keeping imperceptibility. Finally, according to the video encoding framework, we overwrite the frame difference of the frame pair by the watermark to resist compression and transcoding attacks. Furthermore, we propose two optional methods for compensating the special cases of rotation and scaling attacks during watermark detection. Experiments demonstrate the advantage of our method over the existing robust watermarking methods.

Comparative study of 360° virtual reality and traditional two-dimensional video in nonface-to-face dental radiology classes: focusing on learning satisfaction and self-efficacy

BackgroundAcquiring adequate theoretical knowledge in the field of dental radiography (DR) is essential for establishing a good foundation at the prepractical stage. Currently, nonface-to-face DR education predominantly relies on two-dimensional (2D) videos, highlighting the need for developing educational resources that address the inherent limitations of this method. We developed a virtual reality (VR) learning medium using 360° video with a prefabricated head-mounted display (pHMD) for nonface-to-face DR learning and compared it with a 2D video medium.MethodsForty-four participants were randomly assigned to a control group (n = 23; 2D video) and an experimental group (n = 21; 360° VR). DR was re-enacted by the operator and recorded using 360° video. A survey was performed to assess learning satisfaction and self-efficacy. The nonparametric statistical tests comparing the groups were conducted using SPSS statistical analysis software.ResultsLearners in the experimental group could experience VR for DR by attaching their smartphones to the pHMD. The 360° VR video with pHMD provided a step-by-step guide for DR learning from the point of view of an operator as VR. Learning satisfaction and self-efficacy were statistically significantly higher in the experimental group than the control group (p < 0.001).ConclusionsThe 360° VR videos were associated with greater learning satisfaction and self-efficacy than conventional 2D videos. However, these findings do not necessarily substantiate the educational effects of this medium, but instead suggest that it may be considered a suitable alternative for DR education in a nonface-to-face environment. However, further examination of the extent of DR knowledge gained in a nonface-to-face setting is warranted. Future research should aim to develop simulation tools based on 3D objects and also explore additional uses of 360° VR videos as prepractical learning mediums.

Scrutinization of Video posts on Social Media for Authenticity

Multimedia posts, especially video posts are easy to understand, attractive and are considered as proof of evidence for an event that occurred. Videos are one of the frequently shared posts on social media. Videos and images speak louder than words. They are very much believed by people and forwarded. Videos reach every person despite of language or literacy barriers. Technological improvements lead to easy generation of fraudulent videos to create a negative impact on people and society. Celebrities and politicians are highly affected because of fake video generation and dissemination. The video posts uploaded and forwarded on social media should be analyzed and identified to stop propagation before they create harm to the society. The proposed work converts the video clips into a sequence of frames. The keyframes are then identified by the use of histogram comparison. A CNN model is built with optimum layers for image classification. The keyframes or images are then classified using the CNN model to identify fraudulent content. The proposed work is light in terms of processing when compared to existing or conventional video classification. The work uses FaceForensics++, DeepFake and Celeb-DF V2 datasets and achieved 99.7%, 99.8% and 98.01% accuracy to identify fraudulent video posts.

MACHINE LEARNING-BASED FACIAL RECOGNITION FOR VIDEO SURVEILLANCE SYSTEMS

Video surveillance systems play a crucial role in ensuring public safety and security. However, the traditional methods of surveillance often fall short in effectively identifying individuals, particularly in crowded or dynamic environments. This research addresses the limitations of conventional video surveillance by proposing a machine learning-based facial recognition system. The increasing demand for robust security measures necessitates the development of advanced technologies in video surveillance. Facial recognition has emerged as a promising solution, but existing systems struggle with accuracy and efficiency. This research aims to bridge these gaps by leveraging machine learning techniques for facial recognition in video surveillance. Conventional video surveillance struggles with accurate and rapid identification of individuals, leading to potential security lapses. This research addresses the challenge of enhancing facial recognition accuracy in real-time video feeds, especially in scenarios with varying lighting conditions and occlusions. While facial recognition has gained traction, there is a significant research gap in the implementation of machine learning algorithms tailored for video surveillance. This study aims to fill this void by proposing a novel methodology that combines deep learning and computer vision techniques for robust facial recognition in dynamic environments. The proposed methodology involves training a deep neural network on a diverse dataset of facial images to enable the model to learn intricate facial features. Additionally, computer vision algorithms will be employed to handle challenges such as occlusions and varying lighting conditions. The model''''s performance will be evaluated using real-world video surveillance data. Preliminary results demonstrate a significant improvement in facial recognition accuracy compared to traditional methods. The machine learning-based system exhibits enhanced performance in challenging scenarios, showcasing its potential for practical implementation in video surveillance systems.

Conventional videolaryngoscope versus 3D printed videolaryngoscope

&lt;b&gt;Background and objectives:&lt;/b&gt; In patients with Covid-19, using a video laryngoscope as an alternative to direct laryngoscopy is recommended to protect the intubator from infection and reduce intubation failures due to personal protective equipment. The cost of video laryngoscopes limits their availability in all healthcare institutions. The present study aimed to compare the efficacy and safety of 3D printed video laryngoscope and conventional video laryngoscope on intubation.&lt;br /&gt; &lt;b&gt;Material and methods:&lt;/b&gt; 30 ASA I-II patients who were not considered to have a difficult airway were included in the study after obtaining the ethics committee approval from Adnan Menderes University Clinical Research Ethics Committee. Patients were randomly divided into two groups, group 1 and group 2. After the induction of anesthesia under standard monitoring, the Cormack Lehane score was recorded by direct laryngoscopy in all patients. Patients in group 1 were intubated with a 3D-printed video laryngoscope. In contrast, patients in group 2 were intubated with a conventional video laryngoscope (STORZ C-mac videolaryngoscope). Intubation time, number of attempts, and hemodynamic values of patients with early postoperative complications were recorded. The data were recorded and statistically evaluated.&lt;br /&gt; &lt;b&gt;Results:&lt;/b&gt; There were no significant differences between the groups regarding demographic data, BMI, and hemodynamic data. The Cormack Lehane score was calculated as 1.6±0.51 in group 1 and 1.4±0.51 in group 2 (p=0.38). Intubation times of the groups were 32.6±18 s and 27.06±11.37 s, respectively (p=0.4). The number of intubation attempts was 1.2±0.63 in group 1 and 1±0.01 in group 2 (p=0.31). The image quality of the camera by the intubator, intubation conditions, and intubation satisfaction was similar in the two groups.&lt;br /&gt; &lt;b&gt;Conclusion:&lt;/b&gt;&lt;b&gt; &lt;/b&gt;Comparing a 3D-printed videolaryngoscope with a conventional videolaryngoscope, no differences were observed in intubation times, number of intubation attempts, hemodynamic changes, and early postoperative complications. Intubation satisfaction values by the practitioner were found to be similar. It was concluded that the 3D-printed videolaryngoscope, which is cost-effective and easy to access, can be used instead of conventional videolaryngoscope in patients with a normal airway.

HCMS: Hierarchical and Conditional Modality Selection for Efficient Video Recognition

Videos are multimodal in nature. Conventional video recognition pipelines typically fuse multimodal features for improved performance. However, this is not only computationally expensive but also neglects the fact that different videos rely on different modalities for predictions. This article introduces Hierarchical and Conditional Modality Selection (HCMS), a simple yet efficient multimodal learning framework for efficient video recognition. HCMS operates on a low-cost modality, i.e., audio clues, by default, and dynamically decides on-the-fly whether to use computationally expensive modalities, including appearance and motion clues, on a per-input basis. This is achieved by the collaboration of three LSTMs that are organized in a hierarchical manner. In particular, LSTMs that operate on high-cost modalities contain a gating module, which takes as inputs lower-level features and historical information to adaptively determine whether to activate its corresponding modality; otherwise, it simply reuses historical information. We conduct extensive experiments on two large-scale video benchmarks, FCVID and ActivityNet, and the results demonstrate the proposed approach can effectively explore multimodal information for improved classification performance while requiring much less computation.