Loss Of Visual Information Research Articles

Day2Dark: Pseudo-Supervised Activity Recognition Beyond Silent Daylight

AbstractThis paper strives to recognize activities in the dark, as well as in the day. We first establish that state-of-the-art activity recognizers are effective during the day, but not trustworthy in the dark. The main causes are the limited availability of labeled dark videos to learn from, as well as the distribution shift towards the lower color contrast at test-time. To compensate for the lack of labeled dark videos, we introduce a pseudo-supervised learning scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos to improve an activity recognizer in low light. As the lower color contrast results in visual information loss, we further propose to incorporate the complementary activity information within audio, which is invariant to illumination. Since the usefulness of audio and visual features differs depending on the amount of illumination, we introduce our ‘darkness-adaptive’ audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and Charades demonstrate our proposals are superior to image enhancement, domain adaptation and alternative audio-visual fusion methods, and can even improve robustness to local darkness caused by occlusions. Project page: https://xiaobai1217.github.io/Day2Dark/.

Spontaneous eye blinks in horses (Equus caballus) are modulated by attention

Spontaneous eye blinks are brief closures of both eyelids. The spontaneous eye blink rate (SEBR) exceeds physiological corneal needs and is modulated by emotions and cognitive states, including vigilance and attention, in humans. In several animal species, the SEBR is modulated by stress and antipredator vigilance, which may limit the loss of visual information due to spontaneous eye closing. Here, we investigated whether the SEBR is modulated by attention in the domestic horse (Equus caballus). Our data supported previous studies indicating a tonic SEBR specific to each individual. We also found that, superimposed on a tonic SEBR, phasic changes were induced by cognitive processing. Attention downmodulated the SEBR, with the magnitude of blink inhibition proportional to the degree of attentional selectivity. On the other hand, reward anticipation upregulated the SEBR. Our data also suggested that horses possess the cognitive property of object permanence: they understand that an object that is no longer in their visual field has not ceased to exist. In conclusion, our results suggested that spontaneous eye blinks in horses are modulated by attentional cognitive processing.

Autobiographical memory of blind and sighted early teenagers: Memory accessibility, episodicity and phenomenology

AbstractVisual system is crucial to autobiographical memory. Research tended to show that blind adults may compensate for the loss of visual information in retrieval of their autobiographical memories. Much less is known about how blind children's autobiographical memory develops in the absence of visual information. Using cue‐word methodology, 36 sighted and 33 blind early teenagers were asked to recall memories and subsequently rated phenomenological qualities of their memories. Retrieval latency, the number of prompts provided, episodic and non‐episodic details reported for each memory were coded. In terms of memory accessibility, the blind group recalled comparable number of memories with comparable latency to retrieve memories, but they needed more prompting. Blind participants recalled similar number of episodic details; however, they reported more extraneous details, decreasing specificity. Blind early teenagers reported higher auditory imagery, a propensity to remember events from the first‐person perspective, and a tendency to remember events as coherent stories.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

Show, tell and rectify: Boost image caption generation via an output rectifier

Transformer-based models have excellent performance in capturing the interactions between textual and visual features. However, language bias remains a thorny problem in the image captioning domain, leading to the inconsistency between the generated sentences and the actual images. Existing models focus on preventing the wrong words from being output, with little attention to how to correct them. The problem is that if the current word has not yet been output, the model cannot accurately determine whether it is correct. To address this issue, a Double Decoding Transformer framework is proposed. First, a Rectifier is introduced to correct the output sentences in the absence of a language pre-trained module. In addition, visual features provide powerful guidance for attention distribution and redistribution in the Decoder and the Rectifier of the proposed framework, respectively. Due to the presence of downsampling, information loss in the visual feature extraction process is inevitable. Therefore, a Visual Feature Compensation (VFC) module is proposed to compensate for the loss of visual information as much as possible. Finally, by integrating these two modules into a transformer-based framework, a Double Decoding Transformer – D2 Transformer is built. Extensive experiments on the MSCOCO dataset with the “Karpathy” test set demonstrate the validity of the proposed model.

A new point cloud simplification method for reducing visual distortion

Existing point cloud simplification methods often focus on geometric fidelity while paying little attention to the visual distortion they cause. As a result, the simplified point cloud cannot meet the visualization requirements well. This study proposes a new simplification method that significantly reduces the visual distortion in the simplified point cloud. The proposed method begins by establishing a visual information loss function of the simplified point cloud and minimizing it, resulting in a set of simplified point clouds. Then, a quantification criterion for visual distortion in the simplified point cloud is established. By the criterion, point clouds are projected onto geometric and color feature domains that are closely related to visual effects, and the simplified point cloud with a small visual difference from the original point cloud is generated. The effectiveness of the proposed method has been verified using benchmark datasets and a scanned substation's massive point cloud.

Diving into Clarity: Restoring Underwater Images using Deep Learning

In this paper we propose a learning-based restoration approach to learn the optimal parameters for enhancing the quality of different types of underwater images and apply a set of intensity transformation techniques to process raw underwater images. The methodology comprises two steps. Firstly, a Convolutional Neural Network (CNN) Regression model is employed to learn enhancing parameters for each underwater image type. Trained on a diverse dataset, the CNN captures complex relationships, enabling generalization to various underwater conditions. Secondly, we apply intensity transformation techniques to raw underwater images. These transformations collectively compensate for visual information loss due to underwater degradation, enhancing overall image quality. In order to evaluate the performance of our proposed approach, we conducted qualitative and quantitative experiments using well-known underwater image datasets (U45 and UIEB), and using the proposed challenging dataset composed by 276 underwater images from the Amazon region (AUID). The results demonstrate that our approach achieves an impressive accuracy rate in different underwater image datasets. For U45 and UIEB datasets, regarding PSNR and SSIM quality metrics, we achieved 26.967, 0.847, 27.299 and 0.793, respectively. Meanwhile, the best comparison techniques achieved 26.879, 0.831, 27.157 and 0.788, respectively.

FRC-Net: A Simple Yet Effective Architecture for Low-Light Image Enhancement

Low-light image enhancement (LLIE) aims at refining illumination and restoring the details of low-light images. However, current deep LLIE models still face two crucial issues causing blurred textures and inaccurate illumination: 1) low-quality detail-recovery results due to information loss; 2) complex and even redundant model structure. In this paper, we therefore propose a simple yet effective deep LLIE architecture, termed Full-Resolution Context Network (FRC-Net). To avoid the visual information loss caused by feature scaling, we present a novel full-resolution representation strategy to replace all feature scaling operations, which can prevent the information degradation by making the intermediary features keep the original resolution. The structure of FRC-Net is very simple, which only contains 12 cascaded layers: 7 convolutional layers and 5 newly-designed context attention (CA) modules. The plug-and-play CA module is designed to overcome the limited receptive field caused by shallow structures by learning global context as well as retaining local details. Extensive experiments show that our model obtains better detail-recovery quality over current SOTA methods, with relatively fewer parameters and faster inference speed.

Humans gradually integrate sudden gain or loss of visual information into spatial orientation perception.

Vestibular and visual information is used in determining spatial orientation. Existing computational models of orientation perception focus on the integration of visual and vestibular orientation information when both are available. It is well-known, and computational models capture, differences in spatial orientation perception with visual information or without (i.e., in the dark). For example, during earth vertical yaw rotation at constant angular velocity without visual information, humans perceive their rate of rotation to decay. However, during the same sustained rotation with visual information, humans can continue to more accurately perceive self-rotation. Prior to this study, there was no existing literature on human motion perception where visual information suddenly become available or unavailable during self-motion. Via a well verified psychophysical task, we obtained perceptual reports of self-rotation during various profiles of Earth-vertical yaw rotation. The task involved transitions in the availability of visual information (and control conditions with visual information available throughout the motion or unavailable throughout). We found that when visual orientation information suddenly became available, subjects gradually integrated the new visual information over ~10 seconds. In the opposite scenario (visual information suddenly removed), past visual information continued to impact subject perception of self-rotation for ~30 seconds. We present a novel computational model of orientation perception that is consistent with the experimental results presented in this study. The gradual integration of sudden loss or gain of visual information is achieved via low pass filtering in the visual angular velocity sensory conflict pathway. In conclusion, humans gradually integrate sudden gain or loss of visual information into their existing perception of self-motion.

Multisensory effects of mask wearing on speech intelligibility and the benefit of multilingualism.

Due to the pandemic of the Covid-19 disease, it became common to wear masks on some public spaces. By covering mouth and nose, visual-related speech cues are greatly reduced, while the auditory signal is both distorted and attenuated. The present study aimed to analyze the multisensory effects of mask wearing on speech intelligibility and the differences in these effects between participants who spoke 1, 2 and 3 languages. The study consisted of the presentation of sentences from the SPIN test to 40 participants. Participants were asked to report the perceived sentences. There were four conditions: auditory with mask; audiovisual with mask; auditory without mask; audiovisual without mask. Two sessions were conducted, one week apart, each with the same stimuli but with a different signal-to-noise ratio. Results demonstrated that the use of the mask decreased speech intelligibility, both due to a decrease in the quality of auditory stimuli and due to the loss of visual information. Signal-to-noise ratio largely affects speech intelligibility and higher ratios are needed in mask-wearing conditions to obtain any degree of intelligibility. Those who speak more than one language are less affected by mask wearing, as are younger listeners. Wearing a facial mask reduces speech intelligibility, both due to visual and auditory factors. Older people and people who only speak one language are affected the most.

Altered Eye Movements During Reading With Simulated Central and Peripheral Visual Field Defects.

Although foveal vision provides fine spatial information, parafoveal and peripheral vision are also known to be important for efficient reading behaviors. Here we systematically investigate how different types and sizes of visual field defects affect the way visual information is acquired via eye movements during reading. Using gaze-contingent displays, simulated scotomas were induced in 24 adults with normal or corrected-to-normal vision during a reading task. The study design included peripheral and central scotomas of varying sizes (aperture or scotoma size of 2°, 4°, 6°, 8°, and 10°) and no-scotoma conditions. Eye movements (e.g., forward/backward saccades, fixations, microsaccades) were plotted as a function of either the aperture or scotoma size, and their relationships were characterized by the best fitting model. When the aperture size of the peripheral scotoma decreased below 6° (11 visible letters), there were significant decreases in saccade amplitude and velocity, as well as substantial increases in fixation duration and the number of fixations. Its dependency on the aperture size is best characterized by an exponential decay or growth function in log-linear coordinates. However, saccade amplitude and velocity, fixation duration, and forward/regressive saccades increased more or less linearly with increasing central scotoma size in log-linear coordinates. Our results showed differential impacts of central and peripheral vision loss on reading behaviors while lending further support for the importance of foveal and parafoveal vision in reading. These apparently deviated oculomotor behaviors may in part reflect optimal reading strategies to compensate for the loss of visual information.

An image caption model based on attention mechanism and deep reinforcement learning.

Image caption technology aims to convert visual features of images, extracted by computers, into meaningful semantic information. Therefore, the computers can generate text descriptions that resemble human perception, enabling tasks such as image classification, retrieval, and analysis. In recent years, the performance of image caption has been significantly enhanced with the introduction of encoder-decoder architecture in machine translation and the utilization of deep neural networks. However, several challenges still persist in this domain. Therefore, this paper proposes a novel method to address the issue of visual information loss and non-dynamic adjustment of input images during decoding. We introduce a guided decoding network that establishes a connection between the encoding and decoding parts. Through this connection, encoding information can provide guidance to the decoding process, facilitating automatic adjustment of the decoding information. In addition, Dense Convolutional Network (DenseNet) and Multiple Instance Learning (MIL) are adopted in the image encoder, and Nested Long Short-Term Memory (NLSTM) is utilized as the decoder to enhance the extraction and parsing capability of image information during the encoding and decoding process. In order to further improve the performance of our image caption model, this study incorporates an attention mechanism to focus details and constructs a double-layer decoding structure, which facilitates the enhancement of the model in terms of providing more detailed descriptions and enriched semantic information. Furthermore, the Deep Reinforcement Learning (DRL) method is employed to train the model by directly optimizing the identical set of evaluation indexes, which solves the problem of inconsistent training and evaluation standards. Finally, the model is trained and tested on MS COCO and Flickr 30 k datasets, and the results show that the model has improved compared with commonly used models in the evaluation indicators such as BLEU, METEOR and CIDEr.

Comparative study of postural control in 20-40 years olds and karate athletes using a neuromuscular training device

Postural control helps maintain upright posture and contributes to balance regulation. Increasing neuromuscular deficits in older subjects lead to a deterioration in postural control, which can be improved through neuromuscular training. Assuming that neuromuscular deficits occur at an early age, the postural abilities of 20 to 40 year olds were investigated. In addition, the influence of competitive sports on postural control was examined by including karate squad athletes. For an age comparison, 123 age-homogeneous healthy people between 20 and 40 years were examined. 24 karate squad athletes were compared with the 20 -25 year old healthy subjects. A neuromuscular training device, the Human Body Equilibrium 360, was used to collect data. In the age comparison, there were no significant differences for all seven measurements. The study group was too young to detect deficits. However, some general findings could be published. In the two-legged stance, a significant increase in the fluctuations of the body centre of gravity after eye closure could be seen in all groups. The influence of footedness and handedness could be seen in the one-legged stance and the strength test. Gender-specific differences could be observed. The examination of karate squad athletes showed various training-specific adaptations. For example, karate athletes were better able to compensate for the loss of visual information. Karate athletes also showed a significantly wider gait pattern and a larger stability area. Athletes were able to shift their body centre of gravity further without getting out of balance. The Huber 360 is only conditionally suitable as a rehabilitation and training device, due to the low discriminatory power of the tests in its entire range of applications, as significant differences between the groups are difficult to detect. Individual training specific adaptations through performance-oriented karate training are recognisable and underline the influence of training on postural control. In summary, the use of the Huber 360 to assess postural abilities in age comparisons and to distinguish training specific adaptations requires a differentiated and more selective use of the individual tests. Further studies with older patients with different diseases are planned.

Image Enhancement based on the Histogram Equalization and Multiresolution Discrete Stationary Wavelet Transforms

In recent years, due to the tremendous development that took place on the Internet and its applications in the aspects of human life, the demands of using digital images have also increased dramatically, and that opens up horizons for scientific research in the field of improving the quality of the digital images by removing the noise that caused by to processes that are applied within network transmissions like performing storage, retrieval, and encryption to preserve privacy. All these effects are yielded to reduce image quality and loss of visual information. To surmount this problem, image enhancement methods are used to eliminate the noise while preserving supreme exact details and essential characteristics as much as possible in the digital image. The wavelet image enhancement technique played a critical role in this field by attempts to reduce noise in the image while retaining the vital features of the image due to the capability to separate the image into sub-bands (sub-images) and influence the frequencies of each sub-band separately, where acquiring the original image content is essential to obtaining reliable performance.
 Different enhancement techniques have been realized by many researchers so far. Each technique has its own privileges and shortcomings. In this work, a proposed procedure is presented and effectuated to the image modified by Additive White Gaussian Noise (AWGN). The proposed enhancement operation was achieved using the combination of Histogram Equalization with a two-dimensional stationary discrete wavelet transform (2D-SWT) as a multi-resolution analysis technique in image processing at three levels of decomposition to obtain revised results of the method of noise removing. To distinguish and eliminate noise from affected pixel points in the wavelet domain the 2D-SWT is used based on the hard and soft threshold systems on both high and low frequencies to decrease noise from the noisy image. Then, the multi-level 2D inverse wavelet transform (2D-IWT) is applied to eliminate noise and complete the synthesis of the image by the proposed image enhancement techniques.
 In the end, the performance of the proposed methods has been evaluated by using the Peak Signal to Noise Ratio (PSNR). Experimental measurements determine that the results of the proposed techniques enhanced the PSNR by about (16.16%) with respect to the results of the related works, and the structure of the image quality has also been improved in terms of edges retaining and greater noise elimination.

Reliable metrics‐based linear regression model for multilevel privacy measurement of face instances

Social networking sites have made photo sharing convenient and, consequently, users of those sites frequently share photos. The proliferation of these social images throughout the Internet has inadvertently exposed an increasing number of personally identifiable information, particularly from the visual information of the faces in the images. Most existing methods to de-identify faces results in an excessive loss of visual information. To solve this problem, this paper proposes a reliable metrics-based linear regression model for multilevel privacy measurement of face instances using the size information of face instances. The proposed privacy measurement model provides a novel instance-level-based solution to measure privacy levels. The paper also establishes a scientific relationship between the size information of face instances and their privacy levels, quantifying the degree to which face instances need to be de-identified. Finally, the paper proposes a novel k-Same-DT de-identification method to provide reliable metrics for a linear regression model. It is a real-time k-same-related de-identification algorithm that combines the PCA dimensionality reduction strategy and the Delaunay triangle-based face alignment algorithm.The proposed k-Same-DT method creates a de-identified face with a lower identifiable rate and higher structural similarity, and it can provide reliable de-identification metrics for privacy measurement. Experiments using the classical face dataset demonstrates the effectiveness of the proposed de-identification method. Extensive experiments and surveys on real-world social images were also conducted to verify the proposed measurement model.

Face masks and speaking style affect audio-visual word recognition and memory of native and non-native speech.

Though necessary, protective mask wearing in response to the COVID-19 pandemic presents communication challenges. The present study examines how signal degradation and loss of visual information due to masks affects intelligibility and memory for native and non-native speech. We also test whether clear speech can alleviate perceptual difficulty for masked speech. One native and one non-native speaker of English recorded video clips in conversational speech without a mask and conversational and clear speech with a mask. Native English listeners watched video clips presented in quiet or mixed with competing speech. The results showed that word recognition and recall of speech produced with a mask can be as accurate as without a mask in optimal listening conditions. Masks affected non-native speech processing at easier noise levels than native speech. Clear speech with a mask significantly improved accuracy in all listening conditions. Speaking clearly, reducing noise, and using surgical masks as well as good signal amplification can help compensate for the loss of intelligibility due to background noise, lack of visual cues, physical distancing, or non-native speech. The findings have implications for communication in classrooms and hospitals where listeners interact with teachers and healthcare providers, oftentimes non-native speakers, through their protective barriers.

Viewers Change Eye-Blink Rate by Predicting Narrative Content.

Eye blinks provoke a loss of visual information. However, we are not constantly making conscious decisions about the appropriate moment to blink. The presence or absence of eye blinks also denotes levels of attention. We presented three movies with the exact same narrative but different styles of editing and recorded participants’ eye blinks. We found that moments of increased or decreased eye blinks by viewers coincided with the same content in the different movie styles. The moments of increased eye blinks corresponded to those when the actor leaves the scene and when the movie repeats the same action for a while. The moments of decreased eye blinks corresponded to actions where visual information was crucial to proper understanding of the scene presented. According to these results, viewers’ attention is more related to narrative content than to the style of editing when watching movies.

Functional connectivity density alterations in middle-age retinal detachment patients.

ObjectiveMiddle‐age patients with retinal detachment (RD) exhibit a loss of visual information, and previous studies of functional magnetic resonance imaging (fMRI) have demonstrated abnormal spontaneous activity in the RD brain. Therefore, this study assessed changes in local functional connectivity density (lFCD) and long‐range functional connectivity density (longFCD) in middle‐age RD patients during resting‐state FC.MethodsIn total, 32 middle‐age patients with RD (18 men and 14 women), and 32 age‐, sex‐, and education‐matched normal controls (NCs) (18 men and 14 women) were recruited and underwent functional magnetic resonance examination in the resting state. Two‐sample t test was performed to compare lFCD and longFCD between groups. Receiver operating characteristic (ROC) curves were generated to distinguish middle‐age RD patients from NCs.ResultsCompared with NCs, middle‐age RD patients demonstrated increased lFCD values in the right inferior temporal gyrus, and increased longFCD values in the bilateral inferior frontal gyri, left superior and middle frontal gyrus, bilateral inferior temporal gyri, and left cerebellum posterior lobe. Middle‐age RD patient exhibited decreased lFCD values in the left cuneus, right lingual gyrus, and left middle frontal gyrus. They also had lower longFCD values in the left lingual gyrus and left inferior occipital gyrus. ROC curve analysis showed excellent accuracy of the specific areas under the curve.ConclusionOur results reveal that middle‐age RD patients exhibited variations of binarized lFCD and longFCD in specific brain areas, which provides insight into the pathological mechanism of RD patients with acute visual loss.

The Effect of Media Professionalization on Cognitive Neurodynamics During Audiovisual Cuts.

Experts apply their experience to the proper development of their routine activities. Their acquired expertise or professionalization is expected to help in the development of those recurring tasks. Media professionals spend their daily work watching narrative contents on screens, so learning how they manage visual perception of those contents could be of interest in an increasingly audiovisual society. Media works require not only the understanding of the storytelling, but also the decoding of the formal rules and presentations. We recorded electroencephalographic (EEG) signals from 36 participants (18 media professionals and 18 non-media professionals) while they were watching audiovisual contents, and compared their eyeblink rate and their brain activity and connectivity. We found that media professionals decreased their blink rate after the cuts, suggesting that they can better manage the loss of visual information that blinks entail by sparing them when new visual information is being presented. Cuts triggered similar activation of basic brain processing in the visual cortex of the two groups, but different processing in medial and frontal cortical areas, where media professionals showed a lower activity. Effective brain connectivity occurred in a more organized way in media professionals–possibly due to a better communication between cortical areas that are coordinated for decoding new visual content after cuts.

Research on Volleyball Video Intelligent Description Technology Combining the Long-Term and Short-Term Memory Network and Attention Mechanism.

With the development of computer technology, video description, which combines the key technologies in the field of natural language processing and computer vision, has attracted more and more researchers' attention. Among them, how to objectively and efficiently describe high-speed and detailed sports videos is the key to the development of the video description field. In view of the problems of sentence errors and loss of visual information in the generation of the video description text due to the lack of language learning information in the existing video description methods, a multihead model combining the long-term and short-term memory network and attention mechanism is proposed for the intelligent description of the volleyball video. Through the introduction of the attention mechanism, the model pays much attention to the significant areas in the video when generating sentences. Through the comparative experiment with different models, the results show that the model with the attention mechanism can effectively solve the loss of visual information. Compared with the LSTM and base model, the multihead model proposed in this paper, which combines the long-term and short-term memory network and attention mechanism, has higher scores in all evaluation indexes and significantly improved the quality of the intelligent text description of the volleyball video.