Low-level Cues Research Articles

A dissociation between the effects of expectations and attention in selective visual processing

It is often claimed that probabilistic expectations affect visual perception directly, without mediation by selective attention. However, these claims have been disputed, as effects of expectation and attention are notoriously hard to dissociate experimentally. In this study, we used a new approach to separate expectations from attention. In four experiments (N = 60), participants searched for a target in a rapid serial visual presentation (RSVP) stream and had to identify a digit or a letter defined by a low-level cue (colour or shape). Expectations about the target's alphanumeric category were probabilistically manipulated. Since category membership is a high-level feature and since the target was embedded among many distractors that shared its category, targets from the expected category should not attract attention more than targets from the unexpected category. In the first experiment, these targets were more likely to be identified relative to targets from the unexpected category. Importantly, in the following experiments, we also included behavioural and electrophysiological indices of attentional guidance and engagement. This allowed us to examine whether expectations also modulated these or earlier attentional processes. Results showed that category-based expectations had no modulatory effects on attention, and only affected processing at later encoding-related stages. Alternative interpretation of expectation effects in terms of repetition priming or response bias were also ruled out. These observations provide new evidence for direct attention-independent expectation effects on perception. We suggest that expectations can adjust the threshold required for encoding expectations-congruent information, thereby affecting the speed with which target objects are encoded in working memory.

Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images

Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. While these models have numerous benefits across various sectors, they have also raised concerns about the potential misuse of fake images and cast new pressures on fake image detection. In this work, we pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models. Firstly, we conduct a comprehensive analysis of the performance of contrastive and classification-based visual features, respectively extracted from CLIP-based models and ResNet or ViT-based architectures trained on image classification datasets. Our results demonstrate that fake images share common low-level cues, which render them easily recognizable. Further, we devise a multimodal setting wherein fake images are synthesized by different textual captions, which are used as seeds for a generator. Under this setting, we quantify the performance of fake detection strategies and introduce a contrastive-based disentangling method that lets us analyze the role of the semantics of textual descriptions and low-level perceptual cues. Finally, we release a new dataset, called COCOFake, containing about 1.2M images generated from the original COCO image-caption pairs using two recent text-to-image diffusion models, namely Stable Diffusion v1.4 and v2.0.

The Joint Simon task is not joint for capuchin monkeys

Human cooperation can be facilitated by the ability to create a mental representation of one’s own actions, as well as the actions of a partner, known as action co-representation. Even though other species also cooperate extensively, it is still unclear whether they have similar capacities. The Joint Simon task is a two-player task developed to investigate this action co-representation. We tested brown capuchin monkeys (Sapajus [Cebus] apella), a highly cooperative species, on a computerized Joint Simon task and found that, in line with previous research, the capuchins' performance was compatible with co-representation. However, a deeper exploration of the monkeys’ responses showed that they, and potentially monkeys in previous studies, did not understand the control conditions, which precludes the interpretation of the results as a social phenomenon. Indeed, further testing to investigate alternative explanations demonstrated that our results were due to low-level cues, rather than action co-representation. This suggests that the Joint Simon task, at least in its current form, cannot determine whether non-human species co-represent their partner’s role in joint tasks.

Integration of semantic and coarticulation cues during spoken language comprehension in adults

Recent work examining cue integration across levels of linguistic representation has found that listeners can dynamically integrate some of the lower-level and higher-level cues during spoken language comprehension. However, it is still not well understood how the mechanism of cue integration works. This study investigated how adults (n = 52) process preceding higher-level semantic cues and later low-level coarticulation cues during spoken language comprehension using an eye-tracking paradigm. Participants were tested on sentences that contained a prime (semantically related or semantically unrelated to the target) and a target which had varying coarticulation cues (matching versus mismatching splicing cues). Participants were presented with two pictures (target and competitor) on a screen. Analyses looked at the proportion of looking to the target during the prime and target time windows. Results demonstrate that adults flexibly use both the preceding semantic cues and later coarticulatory cues once they are available. Our findings also indicate that adults flexibly weighed both the preceding higher-level and later lower-level cues, such that the processing of low-level coarticulatory cue varied depending on the semantic context. We have added an unstudied level of cue (semantic context) to the set of cues that our cognitive system can integrate during language comprehension.

The Suite for the Assessment of Low-Level cues on Orientation (SALLO): The psychophysics of spatial orientation in virtual reality

Spatial orientation is a complex ability that emerges from the interaction of several systems in a way that is still unclear. One of the reasons limiting the research on the topic is the lack of methodologies aimed at studying multimodal psychophysics in an ecological manner and with affordable settings. Virtual reality can provide a workaround to this impasse by using virtual stimuli rather than real ones. However, the available virtual reality development platforms are not meant for psychophysical testing; therefore, using them as such can be very difficult for newcomers, especially the ones new to coding. For this reason, we developed SALLO, the Suite for the Assessment of Low-Level cues on Orientation, which is a suite of utilities that simplifies assessing the psychophysics of multimodal spatial orientation in virtual reality. The tools in it cover all the fundamental steps to design a psychophysical experiment. Plus, dedicated tracks guide the users in extending the suite components to simplify developing new experiments. An experimental use-case used SALLO and virtual reality to show that the head posture affects both the egocentric and the allocentric mental representations of spatial orientation. Such a use-case demonstrated how SALLO and virtual reality can be used to accelerate hypothesis testing concerning the psychophysics of spatial orientation and, more broadly, how the community of researchers in the field may benefit from such a tool to carry out their investigations.

An insect brain organizes numbers on a left-to-right mental number line

The "mental number line" (MNL) is a form of spatial numeric representation that associates small and large numbers with the left and right spaces, respectively. This spatio-numeric organization can be found in adult humans and has been related to cultural factors such as writing and reading habits. Yet, both human newborns and birds order numbers consistently with an MNL, thus raising the question of whether culture is a main explanation for MNL. Here, we explored the numeric sense of honey bees and show that after being trained to associate numbers with a sucrose reward, they order numbers not previously experienced from left to right according to their magnitude. Importantly, the location of a number on that scale varies with the reference number previously trained and does not depend on low-level cues present on numeric stimuli. We provide a series of neural explanations for this effect based on the extensive knowledge accumulated on the neural underpinnings of visual processing in honey bees and conclude that the MNL is a form of numeric representation that is evolutionarily conserved across nervous systems endowed with a sense of number, irrespective of their neural complexity.

Infants' preferences for approachers over repulsers shift between 4 and 8 months of age

Despite its adaptive value for social life, the emergence and the development of the ability to detect agents that cause aversive interactions and distinguish them from potentially affiliative agents (approachers) has not been investigated. We presented infants with a simple interaction involving two agents: one of them (the "repulser") moved toward and pushed the other (the "approacher") which reacted by simply moving toward the repulser without contacting it. We found that 8-month-olds (N = 28) looked longer at the approacher than at the repulser (Experiment 1), whereas 4-month-olds (N = 30) exhibited no preference (Experiment 2). To control for low-level cues (such as the preference for the agent that moved after the contact), two new groups of 4- and 8-month-old infants were presented with a series of interactions in which the agents inverted their social roles. Older infants (N = 30) manifested no preference for either agent (Experiment 3), while younger infants (N = 30) looked longer at the first agent to move (Experiment 4). Our results indicated that 8-month-olds' preferences for the approacher over the repulser depended on social information and were finely tuned to agents that display prosocial rather than antisocial behavior. We discuss these findings in light of the development and adaptive value of the ability to negatively evaluate repulsers, to avoid choosing them as partners.

What you hear first, is what you get: Initial metrical cue presentation modulates syllable detection in sentence processing.

Auditory rhythms create powerful expectations for the listener. Rhythmic cues with the same temporal structure as subsequent sentences enhance processing compared with irregular or mismatched cues. In the present study, we focus on syllable detection following matched rhythmic cues. Cues were aligned with subsequent sentences at the syllable (low-level cue) or the accented syllable (high-level cue) level. A different group of participants performed the task without cues to provide a baseline. We hypothesized that unaccented syllable detection would be faster after low-level cues, and accented syllable detection would be faster after high-level cues. There was no difference in syllable detection depending on whether the sentence was preceded by a high-level or low-level cue. However, the results revealed a priming effect of the cue that participants heard first. Participants who heard a high-level cue first were faster to detect accented than unaccented syllables, and faster to detect accented syllables than participants who heard a low-level cue first. The low-level-first participants showed no difference between detection of accented and unaccented syllables. The baseline experiment confirmed that hearing a low-level cue first removed the benefit of the high-level grouping structure for accented syllables. These results suggest that the initially perceived rhythmic structure influenced subsequent cue perception and its influence on syllable detection. Results are discussed in terms of dynamic attending, temporal context effects, and implications for context effects in neural entrainment.

Context-Dependent Object Proposal and Recognition

Accurate and fast object recognition is crucial in applications such as automatic driving and unmanned aerial vehicles. Traditional object recognition methods relying on image-wise computations cannot afford such real-time applications. Object proposal methods appear to fit into this scenario by segmenting object-like regions to be further analyzed by sophisticated recognition models. Traditional object proposal methods have the drawback of generating many proposals in order to maintain a satisfactory recall of true objects. This paper presents two proposal refinement strategies based on low-level cues and context-dependent features, respectively. The low-level cues are used to enhance the edge image, while the context-dependent features are verified to rule out false objects that are irrelevant to our application. In particular, the context of the drink commodity is considered because the drink commodity has the largest sales in Taiwan’s convenience store chains, and the analysis of its context has great value in marketing and management. We further developed a support vector machine (SVM) based on the Bag of Words (BoW) model with scale-invariant feature transform (SIFT) descriptors to recognize the proposals. The experimental results show that our object proposal method generates many fewer proposals than those generated by Selective Search and EdgeBoxes, with similar recall. For the performance of SVM, at least 82% of drink objects are correctly recognized for test datasets of various challenging difficulties.

A Novel Multi-Focus Image Fusion Network with U-Shape Structure.

Multi-focus image fusion has become a very practical image processing task. It uses multiple images focused on various depth planes to create an all-in-focus image. Although extensive studies have been produced, the performance of existing methods is still limited by the inaccurate detection of the focus regions for fusion. Therefore, in this paper, we proposed a novel U-shape network which can generate an accurate decision map for the multi-focus image fusion. The Siamese encoder of our U-shape network can preserve the low-level cues with rich spatial details and high-level semantic information from the source images separately. Moreover, we introduce the ResBlocks to expand the receptive field, which can enhance the ability of our network to distinguish between focus and defocus regions. Moreover, in the bridge stage between the encoder and decoder, the spatial pyramid pooling is adopted as a global perception fusion module to capture sufficient context information for the learning of the decision map. Finally, we use a hybrid loss that combines the binary cross-entropy loss and the structural similarity loss for supervision. Extensive experiments have demonstrated that the proposed method can achieve the state-of-the-art performance.

Image Deblurring Using Multi-Stream Bottom-Top-Bottom Attention Network and Global Information-Based Fusion and Reconstruction Network.

Image deblurring has been a challenging ill-posed problem in computer vision. Gaussian blur is a common model for image and signal degradation. The deep learning-based deblurring methods have attracted much attention due to their advantages over the traditional methods relying on hand-designed features. However, the existing deep learning-based deblurring techniques still cannot perform well in restoring the fine details and reconstructing the sharp edges. To address this issue, we have designed an effective end-to-end deep learning-based non-blind image deblurring algorithm. In the proposed method, a multi-stream bottom-top-bottom attention network (MBANet) with the encoder-to-decoder structure is designed to integrate low-level cues and high-level semantic information, which can facilitate extracting image features more effectively and improve the computational efficiency of the network. Moreover, the MBANet adopts a coarse-to-fine multi-scale strategy to process the input images to improve image deblurring performance. Furthermore, the global information-based fusion and reconstruction network is proposed to fuse multi-scale output maps to improve the global spatial information and recurrently refine the output deblurred image. The experiments were done on the public GoPro dataset and the realistic and dynamic scenes (REDS) dataset to evaluate the effectiveness and robustness of the proposed method. The experimental results show that the proposed method generally outperforms some traditional deburring methods and deep learning-based state-of-the-art deblurring methods such as scale-recurrent network (SRN) and denoising prior driven deep neural network (DPDNN) in terms of such quantitative indexes as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) and human vision.

Capitalization interacts with syntactic complexity.

We investigated whether readers use the low-level cue of proper noun capitalization in the parafovea to infer syntactic category, and whether this results in an early update of the representation of a sentence's syntactic structure. Participants read sentences containing either a subject relative or object relative clause, in which the relative clause's overt argument was a proper noun (e.g., The tall lanky guard who alerted Charlie/Charlie alerted to the danger was young) across three experiments. In Experiment 1 these sentences were presented in normal sentence casing or entirely in upper case. In Experiment 2 participants received either valid or invalid parafoveal previews of the relative clause. In Experiment 3 participants viewed relative clauses in only normal conditions. We hypothesized that we would observe relative clause effects (i.e., inflated fixation times for object relative clauses) while readers were still fixated on the word who, if readers use capitalization to infer a parafoveal word's syntactic class. This would constitute a syntactic parafoveal-on-foveal effect. Furthermore, we hypothesized that this effect should be influenced by sentence casing in Experiment 1 (with no cue for syntactic category being available in upper case sentences) but not by parafoveal preview validity of the target words. We observed syntactic parafoveal-on-foveal effects in Experiment 1 and 3, and a Bayesian analysis of the combined data from all three experiments. These effects seemed to be influenced more by noun capitalization than lexical processing. We discuss our findings in relation to models of eye movement control and sentence processing theories. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Implied motion as a possible mechanism for encoding other people's attention.

Recent evidence suggests that the human brain automatically constructs a rich model of other people's attention, beyond registering low-level cues such as someone else's gaze direction. This model is not a physically accurate representation of attention, but instead appears to contain simplifying and physically incoherent features. For example, without explicitly realizing it, people treat the attentive gaze of others as though it exerts a gentle force pushing on objects. Here we specify another aspect of that implicit model of attention. People treat the attentive gaze of an agent as though it were travelling through space, with an implied motion encoded literally enough that it causes a perceptual motion adaptation effect. This implicit model of other people's attention may facilitate the process of keeping track of who is attending to what, which is essential for reading and predicting the minds and behavior of social agents. This implicit model of attention may also have shaped culturally widespread ideas about mind and spirit.

Phonetic and phonological effects of tonal information in the segmentation of Korean speech: An artificial-language segmentation study

AbstractThis study investigates how the fine-grained phonetic realization of tonal cues impacts speech segmentation when the cues signal the same word boundary in the native and unfamiliar languages but do so differently. Korean listeners use the phrase-final high (H) tone and the phrase-initial low (L) tone to segment speech into words (Kim, Broersma, & Cho, 2012; Kim & Cho, 2009), but it is unclear how the alignment of the phrase-final H tone and the scaling of the phrase-initial L tone modulate their speech segmentation. Korean listeners completed three artificial-language (AL) tasks (within-subject): (a) one AL without tonal cues; (b) one AL with later-aligned phrase-final H cues (non-Korean-like); and (c) one AL with earlier-aligned phrase-final H cues (Korean-like). Three groups of Korean listeners heard (b) and (c) in three phrase-initial L scaling conditions (between-subject): high (non-Korean-like), mid (non-Korean-like), or low (Korean-like). Korean listeners’ segmentation improved as the L tone was lowered, and (b) enhanced segmentation more than (c) in the high- and mid-scaling conditions. We propose that Korean listeners tune in to low-level cues (the greater H-to-L slope in [b]) that conform to the Korean intonational grammar when the phrase-initial L tone is not canonical phonologically.

Automatic Catheter and Tube Detection in Pediatric X-ray Images Using a Scale-Recurrent Network and Synthetic Data.

Catheters are commonly inserted life supporting devices. Because serious complications can arise from malpositioned catheters, X-ray images are used to assess the position of a catheter immediately after placement. Previous computer vision approaches to detect catheters on X-ray images were either rule-based or only capable of processing a limited number or type of catheters projecting over the chest. With the resurgence of deep learning, supervised training approaches are beginning to show promising results. However, dense annotation maps are required, and the work of a human annotator is difficult to scale. In this work, we propose an automatic approach for detection of catheters and tubes on pediatric X-ray images. We propose a simple way of synthesizing catheters on X-ray images to generate a training dataset by exploiting the fact that catheters are essentially tubular structures with various cross sectional profiles. Further, we develop a UNet-style segmentation network with a recurrent module that can process inputs at multiple scales and iteratively refine the detection result. By training on adult chest X-rays, the proposed network exhibits promising detection results on pediatric chest/abdomen X-rays in terms of both precision and recall, with Fβ = 0.8. The approach described in this work may contribute to the development of clinical systems to detect and assess the placement of catheters on X-ray images. This may provide a solution to triage and prioritize X-ray images with potentially malpositioned catheters for a radiologist's urgent review and help automate radiology reporting.

Saliency Detection Based on the Combination of High-Level Knowledge and Low-Level Cues in Foggy Images

A key issue in saliency detection of the foggy images in the wild for human tracking is how to effectively define the less obvious salient objects, and the leading cause is that the contrast and resolution is reduced by the light scattering through fog particles. In this paper, to suppress the interference of the fog and acquire boundaries of salient objects more precisely, we present a novel saliency detection method for human tracking in the wild. In our method, a combination of object contour detection and salient object detection is introduced. The proposed model can not only maintain the object edge more precisely via object contour detection, but also ensure the integrity of salient objects, and finally obtain accurate saliency maps of objects. Firstly, the input image is transformed into HSV color space, and the amplitude spectrum (AS) of each color channel is adjusted to obtain the frequency domain (FD) saliency map. Then, the contrast of the local-global superpixel is calculated, and the saliency map of the spatial domain (SD) is obtained. We use Discrete Stationary Wavelet Transform (DSWT) to fuse the cues of the FD and SD. Finally, a fully convolutional encoder–decoder model is utilized to refine the contour of the salient objects. Experimental results demonstrate that the presented model can remove the influence of fog efficiently, and the performance is better than 16 state-of-the-art saliency models.

Image segmentation driven by elements of form

While luminance, contrast, orientation, and terminators are well-established features that are extracted in early visual processing and support the parsing of an image into its component regions, the role of more complex features, such as closure and convexity, is less clear. A main barrier in understanding the roles of such features is that manipulating their occurrence typically entails changes in the occurrence of more elementary features as well. To address this problem, we developed a set of synthetic visual textures, constructed by replacing the binary coloring of standard maximum-entropy textures with tokens (tiles) containing curved or angled elements. The tokens were designed so that there were no discontinuities at their edges, and so that changing the correlation structure of the underlying binary texture changed the shapes that were produced. The resulting textures were then used in psychophysical studies, demonstrating that the resulting feature differences sufficed to drive segmentation. However, in contrast to previous findings for lower-level features, sensitivities to increases and decreases of feature occurrence were unequal. Moreover, the texture-segregation response depended on the kind of token (curved vs. angular, filled-in vs. outlined), and not just on the correlation structure. Analysis of this dependence indicated that simple closed contours and convex elements suffice to drive image segmentation, in the absence of changes in lower-level cues.

A propagation-DNN: Deep combination learning of multi-level features for MR prostate segmentation

Background and objectiveProstate segmentation on Magnetic Resonance (MR) imaging is problematic because disease changes the shape and boundaries of the gland and it can be difficult to separate the prostate from surrounding tissues. We propose an automated model that extracts and combines multi-level features in a deep neural network to segment prostate on MR images. MethodsOur proposed model, the Propagation Deep Neural Network (P-DNN), incorporates the optimal combination of multi-level feature extraction as a single model. High level features from the convolved data using DNN are extracted for prostate localization and shape recognition, while labeling propagation, by low level cues, is embedded into a deep layer to delineate the prostate boundary. ResultsA well-recognized benchmarking dataset (50 training data and 30 testing data from patients) was used to evaluate the P-DNN. When compared it to existing DNN methods, the P-DNN statistically outperformed the baseline DNN models with an average improvement in the DSC of 3.19%. When compared to the state-of-the-art non-DNN prostate segmentation methods, P-DNN was competitive by achieving 89.9 ± 2.8% DSC and 6.84 ± 2.5 mm HD on training sets and 84.13 ± 5.18% DSC and 9.74 ± 4.21 mm HD on testing sets. ConclusionOur results show that P-DNN maximizes multi-level feature extraction for prostate segmentation of MR images.

Colourization of Grayscale Images Using Deep Learning

We propose a deep learning approach for image colourization. The system directly maps a grayscale image to an output colourization using a neural network. The network propagates through low level cues and high level semantics information, which it learns from large scale dataset. One can easily find similar projects present in the web, but in our project we will be having a framework which can incorporate user hints, as to give desired colourization. We know that in a black-white image a red object and a blue object will appear similar of the shade of black. Thus many a time the colour put onto the object by the algorithm might not be similar to what we are used of seeing around. In this case history and prior knowledge of the image is necessary for accuracy and helps the user to get image which is in accordance to the perception of the user for that image. The neural network which will be taught to add colour to a grey scale image with the right gradient of colour for specific objects based on nature. We will be providing simple images to the neural network at first so as to identify simple objects and understand the colours involved with those objects. Thus when we pass on an image comprising of various different objects, then the neural network is responsible of identifying each of the unique objects in that image and add colour to it. The key features of this project would be to get a medium-tohigh level accuracy of colourization of a given image. The algorithm would add colour to an image in real time without much latency and delay involved.

Reconstructing Hand Poses Using Visible Light

Free-hand gestural input is essential for emerging user interactions. We present Aili, a table lamp reconstructing a 3D hand skeleton in real time, requiring neither cameras nor on-body sensing devices. Aili consists of an LED panel in a lampshade and a few low-cost photodiodes embedded in the lamp base. To reconstruct a hand skeleton, Aili combines 2D binary blockage maps from vantage points of different photodiodes, which describe whether a hand blocks light rays from individual LEDs to all photodiodes. Empowering a table lamp with sensing capability, Aili can be seamlessly integrated into the existing environment. Relying on such low-level cues, Aili entails lightweight computation and is inherently privacy-preserving. We build and evaluate an Aili prototype. Results show that Aili’s algorithm reconstructs a hand pose within 7.2 ms on average, with 10.2° mean angular deviation and 2.5-mm mean translation deviation in comparison to Leap Motion. We also conduct user studies to examine the privacy issues of Leap Motion and solicit feedback on Aili’s privacy protection. We conclude by demonstrating various interaction applications Aili enables.