Visual Reasoning Tasks Research Articles

Differential Involvement of EEG Oscillatory Components in Sameness versus Spatial-Relation Visual Reasoning Tasks.

The development of deep convolutional neural networks (CNNs) has recently led to great successes in computer vision, and CNNs have become de facto computational models of vision. However, a growing body of work suggests that they exhibit critical limitations on tasks beyond image categorization. Here, we study one such fundamental limitation, concerning the judgment of whether two simultaneously presented items are the same or different (SD) compared with a baseline assessment of their spatial relationship (SR). In both human subjects and artificial neural networks, we test the prediction that SD tasks recruit additional cortical mechanisms which underlie critical aspects of visual cognition that are not explained by current computational models. We thus recorded electroencephalography (EEG) signals from human participants engaged in the same tasks as the computational models. Importantly, in humans the two tasks were matched in terms of difficulty by an adaptive psychometric procedure; yet, on top of a modulation of evoked potentials (EPs), our results revealed higher activity in the low β (16–24 Hz) band in the SD compared with the SR conditions. We surmise that these oscillations reflect the crucial involvement of additional mechanisms, such as working memory and attention, which are missing in current feed-forward CNNs.

Looking with the (computer) mouse: How to unveil problem-solving strategies in matrix reasoning without eye-tracking.

Problem-solving strategies in visual reasoning tasks are often studied based on the analysis of eye movements, which yields high-quality data but is costly and difficult to implement on a large scale. We devised a new graphical user interface for matrix reasoning tasks where the analysis of computer mouse movements makes it possible to investigate item exploration and, in turn, problem-solving strategies. While relying on the same active perception principles underlying eye-tracking (ET) research, this approach has the additional advantages of being user-friendly and easy to implement in real-world testing conditions, and records only voluntary decisions. A pilot study confirmed that embedding items of Raven's Advanced Progressive Matrices (APM) in the interface did not significantly alter its psychometric properties. Experiment 1 indicated that mouse-based exploration indices, when used to assess two major problem-solving strategies in the APM, are related to final performance-as has been found in past ET research. Experiment 2 suggested that constraining some features of the interface favored the adoption of the more efficient solving strategy for some participants. Overall, the findings support the relevance of the present methodology for accessing and manipulating problem-solving strategies.

Multi-Level Knowledge Injecting for Visual Commonsense Reasoning

When glancing at an image, human can infer what is hidden in the image beyond what is visually obvious, such as objects' functions, people's intents and mental states. However, such a visual reasoning paradigm is tremendously difficult for computer, requiring knowledge about how the world works. To address this issue, we propose Commonsense Knowledge based Reasoning Model (CKRM) to acquire external knowledge to support Visual Commonsense Reasoning (VCR) task, where the computer is expected to answer challenging visual questions. Our key ideas are: (1) To bridge the gap between recognition-level and cognition-level image understanding, we inject external commonsense knowledge via multi-level knowledge transfer network, achieving cell-level, layer-level and attention-level joint information transfer. It can effectively capture knowledge from different perspectives, and perceive common sense of human in advance. (2) To further promote image understanding at cognitive level, we propose a knowledge based reasoning approach, which can relate the transferred knowledge to visual content and compose the reasoning cues to derive the final answer. Experiments are conducted on the challenging visual commonsense reasoning dataset VCR to verify the effectiveness of our proposed CKRM approach, which can significantly improve reasoning performance and achieve the state-of-the-art accuracy.

Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning

As a comprehensive indicator of mathematical thinking and intelligence, the number sense (Dehaene 2011) bridges the induction of symbolic concepts and the competence of problem-solving. To endow such a crucial cognitive ability to machine intelligence, we propose a dataset, Machine Number Sense (MNS), consisting of visual arithmetic problems automatically generated using a grammar model—And-Or Graph (AOG). These visual arithmetic problems are in the form of geometric figures: each problem has a set of geometric shapes as its context and embedded number symbols. Solving such problems is not trivial; the machine not only has to recognize the number, but also to interpret the number with its contexts, shapes, and relations (e.g., symmetry) together with proper operations. We benchmark the MNS dataset using four predominant neural network models as baselines in this visual reasoning task. Comprehensive experiments show that current neural-network-based models still struggle to understand number concepts and relational operations. We show that a simple brute-force search algorithm could work out some of the problems without context information. Crucially, taking geometric context into account by an additional perception module would provide a sharp performance gain with fewer search steps. Altogether, we call for attention in fusing the classic search-based algorithms with modern neural networks to discover the essential number concepts in future research.

Testing the visual impedance effect in children with and without reading difficulties using a new visual reasoning task.

This study examined reasoning skills in children, specifically transitive reasoning and the visual impedance effect, with a new visual/pictorial task. The visual impedance effect is the effect produced by the possible interference in the reasoning process of irrelevant details elicited from the premises of a reasoning task. The new task had no reading requirements, which made it suitable for testing reasoning in primary school children, especially children with reading difficulties (RD), such as dyslexia. The study aimed also to validate the possible use of the task for studying reasoning and detecting the visual impedance effect without the interference of reading skills and to investigate the association between transitive reasoning and reading abilities. A pilot study (N=10) was used to test the suitability of the new task for primary school children. Afterwards, the task was tested on a larger sample of children of third to sixth Grade, with and without RD (N=84). Results showed that the new task is able to detect the main reasoning effects as well as the visual impedance effect. The findings are discussed, with the new task considered appropriate for studying reasoning skills in child populations both with and without RD.

An Exploratory User Study of Visual Causality Analysis

AbstractInteractive visualization tools are being used by an increasing number of members of the general public; however, little is known about how, and how well, people use visualizations to infer causality. Adapted from the mediation causal model, we designed an analytic framework to systematically evaluate human performance, strategies, and pitfalls in a visual causal reasoning task. We recruited 24 participants and asked them to identify the mediators in a fictitious dataset using bar charts and scatter plots within our visualization interface. The results showed that the accuracy of their responses as to whether a variable is a mediator significantly decreased when a confounding variable directly influenced the variable being analyzed. Further analysis demonstrated how individual visualization exploration strategies and interfaces might influence reasoning performance. We also identified common strategies and pitfalls in their causal reasoning processes. Design implications for how future visual analytics tools can be designed to better support causal inference are discussed.

Impact of processing load on analogical mapping with visual sequences in children with developmental language disorders (DLD).

Analogical mapping is a domain-general cognitive process used notably in language development, and particularly in the abstraction of construction schemas. Children with developmental language disorders (DLD) display an impairment in linguistic productivity and creativity, which can be linked to a lack of generalization of construction schemas. To investigate analogical mapping in children with DLD, and especially the influence of processing load, as it could explain the lack of creativity observed in children with DLD. It was hypothesized that analogical mapping is altered in children with DLD and that greater cognitive load (sequential presentation and no perceptual support) would be linked to poorer performance in these children. Fifteen children with DLD and their age-matched peers were administrated a visual analogical reasoning task where they had to complete a sequence sharing the same relational structure as previously presented sequences. Two factors influencing processing load were studied: the modality of presentation (sequential versus simultaneous) and the perceptual support (with versus without). Results showed an expected group effect with poorer performance in children with DLD compared with children with typical language development (TLD). Results corroborated hypotheses according to which children with DLD have difficulties with analogical mapping, which could hinder their abstraction of construction schemas. Results about the influence of processing load were mixed. While the difference between the two groups was more marked for the items without perceptual support than for those with perceptual support, children with DLD were not more affected by the sequential presentation than children with TLD. Children with DLD have impaired analogical mapping competences, especially when the relational similarities are not supported by perceptual cues. This impairment may be the cause of their difficulties in abstracting construction schemas, thus provoking their poor linguistic productivity and creativity. However, more studies are needed to confirm this hypothesis, as the influence of analogical reasoning on language development could also be reversed or could be linked to another external factor.

FiLM: Visual Reasoning with a General Conditioning Layer

We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - answering image-related questions which require a multi-step, high-level process - a task which has proven difficult for standard deep learning methods that do not explicitly model reasoning. Specifically, we show on visual reasoning tasks that FiLM layers 1) halve state-of-the-art error for the CLEVR benchmark, 2) modulate features in a coherent manner, 3) are robust to ablations and architectural modifications, and 4) generalize well to challenging, new data from few examples or even zero-shot.

The Contribution of Increased Gamma Band Connectivity to Visual Non-Verbal Reasoning in Autistic Children: A MEG Study.

Some individuals with autism spectrum (AS) perform better on visual reasoning tasks than would be predicted by their general cognitive performance. In individuals with AS, mechanisms in the brain’s visual area that underlie visual processing play a more prominent role in visual reasoning tasks than they do in normal individuals. In addition, increased connectivity with the visual area is thought to be one of the neural bases of autistic visual cognitive abilities. However, the contribution of such brain connectivity to visual cognitive abilities is not well understood, particularly in children. In this study, we investigated how functional connectivity between the visual areas and higher-order regions, which is reflected by alpha, beta and gamma band oscillations, contributes to the performance of visual reasoning tasks in typically developing (TD) (n = 18) children and AS children (n = 18). Brain activity was measured using a custom child-sized magneto-encephalograph. Imaginary coherence analysis was used as a proxy to estimate the functional connectivity between the occipital and other areas of the brain. Stronger connectivity from the occipital area, as evidenced by higher imaginary coherence in the gamma band, was associated with higher performance in the AS children only. We observed no significant correlation between the alpha or beta bands imaginary coherence and performance in the both groups. Alpha and beta bands reflect top-down pathways, while gamma band oscillations reflect a bottom-up influence. Therefore, our results suggest that visual reasoning in AS children is at least partially based on an enhanced reliance on visual perception and increased bottom-up connectivity from the visual areas.

Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objects

Human vision greatly benefits from the information about sizes of objects. The role of size in several visual reasoning tasks has been thoroughly explored in human perception and cognition. However, the impact of the information about sizes of objects is yet to be determined in AI. We postulate that this is mainly attributed to the lack of a comprehensive repository of size information. In this paper, we introduce a method to automatically infer object sizes, leveraging visual and textual information from web. By maximizing the joint likelihood of textual and visual observations, our method learns reliable relative size estimates, with no explicit human supervision. We introduce the relative size dataset and show that our method outperforms competitive textual and visual baselines in reasoning about size comparisons.

Visual fixation in human newborns correlates with extensive white matter networks and predicts long-term neurocognitive development.

Infants are well known to seek eye contact, and they prefer to fixate on developmentally meaningful objects, such as the human face. It is also known, that visual abilities are important for the developmental cascades of cognition from later infancy to childhood. It is less understood, however, whether newborn visual abilities relate to later cognitive development, and whether newborn ability for visual fixation can be assigned to early microstructural maturation. Here, we investigate relationship between newborn visual fixation (VF) and gaze behavior (GB) to performance in visuomotor and visual reasoning tasks in two cohorts with cognitive follow-up at 2 (n = 57) and 5 (n = 1410) years of age. We also analyzed brain microstructural correlates to VF (n = 45) by voxel-based analysis of fractional anisotropy (FA) in newborn diffusion tensor imaging. Our results show that newborn VF is significantly related to visual-motor performance at both 2 and 5 years, as well as to visual reasoning at 5 years of age. Moreover, good newborn VF relates to widely increased FA levels across the white matter. Comparison to motor performance indicated that early VF is preferentially related to visuocognitive development, and that early motor performance relates neither to white matter integrity nor to visuocognitive development. The present findings suggest that newborn VF is supported by brainwide subcortical networks and it represents an early building block for the developmental cascades of cognition.

Exploring the potential role of visual reasoning tasks among inexperienced solvers

The collective case study described herein explores solution approaches to a task requiring visual reasoning by students and teachers unfamiliar with such tasks. The context of this study is the teaching and learning of calculus in the Palestinian educational system. In the Palestinian mathematics curriculum the roles of visual displays rarely go beyond the illustrative and supplementary, while tasks which demand visual reasoning are absent. In the study, ten teachers and twelve secondary and first year university students were presented with a calculus problem, selected in an attempt to explore visual reasoning on the notions of function and its derivative and how it interrelates with conceptual reasoning. A construct named “visual inferential conceptual reasoning” was developed and implemented in order to analyze the responses. In addition, subjects’ reflections on the task, as well as their attitudes about possible uses of visual reasoning tasks in general, were collected and analyzed. Most participants faced initial difficulties of different kinds while solving the problem; however, in their solution processes various approaches were developed. Reflecting on these processes, subjects tended to agree that such tasks can promote and enhance conceptual understanding, and thus their incorporation in the curriculum would be beneficial.

A custom magnetoencephalography device reveals brain connectivity and high reading/decoding ability in children with autism.

A subset of individuals with autism spectrum disorder (ASD) performs more proficiently on certain visual tasks than may be predicted by their general cognitive performances. However, in younger children with ASD (aged 5 to 7), preserved ability in these tasks and the neurophysiological correlates of their ability are not well documented. In the present study, we used a custom child-sized magnetoencephalography system and demonstrated that preserved ability in the visual reasoning task was associated with rightward lateralisation of the neurophysiological connectivity between the parietal and temporal regions in children with ASD. In addition, we demonstrated that higher reading/decoding ability was also associated with the same lateralisation in children with ASD. These neurophysiological correlates of visual tasks are considerably different from those that are observed in typically developing children. These findings indicate that children with ASD have inherently different neural pathways that contribute to their relatively preserved ability in visual tasks.

Gender Differences in Brain Activation on a Mental Rotation Task

ABSTRACTFew neuroimaging studies have explored gender differences on mental rotation tasks. Most studies have utilized samples with both genders, samples mainly consisting of men, or samples with six or fewer females. Graduate students in science fields or liberal arts programs (20 males, 20 females) completed a mental rotation task during functional magnetic resonance imaging (fMRI). When a pair of cube figures was shown, the participant made a keypad response based on whether the pair is the same/similar or different. Regardless of gender, the bilateral middle frontal gyrus, bilateral intraparietal sulcus (IPS), and the left precuneus were activated when a subject tried to solve the mental rotation task. Increased activation in the right inferior frontal gyrus/middle frontal gyrus, the left precuneus/posterior cingulate cortex/cuneus region, and the left middle occipital gyrus was found for men as compared to women. Better accuracy and shorter response times were correlated with an increased activation in the bilateral intraparietal sulcus. No significant brain activity differences related to mental rotation were found between academic majors. These findings suggest that networks involved in visual attention appear to be more strongly activated in the mental rotation tasks in men as compared to women. It also suggests that men use a more automatic process when analyzing complex visual reasoning tasks while women use a more top-down process.

A case study of Locked-In-Syndrome: Psychological and personal perspectives

Objectives: Although patients with Locked-In-Syndrome (LIS) are often stated to have normal cognitive functioning, the few reports of neuropsychological assessment in these cases suggest this is not always true. This paper (a) reviews published reports of neuropsychological assessments of LIS patients, (b) presents a detailed neuropsychological assessment of a patient to determine if she has normal cognitive functioning and (c) presents the views of the patient on what has happened to her.Methods and procedures: An in-depth single case report assessing the cognitive and emotional functioning of a young woman with LIS plus a personal account from the patient. Detailed assessments were carried out using standardized neuropsychological tests and questionnaires measuring emotional functioning, pain and quality-of-life. The patient also states her views of what happened and how she feels.Results: On most tests, cognitive functioning was average or above, but some impairments were found (consistent with published literature). No emotional problems were detected. Quality of life was satisfactory. Pain perception was normal.Conclusions: Although LIS patients have no severe cognitive deficits, some cognitive difficulties are common. Despite severe physical and communication problems and some mild cognitive deficits on visual reasoning tasks, this patient feels she has a reasonable quality-of-life.

The role of relational reasoning in children's addition concepts

The study addresses the relational reasoning of different‐aged children and how addition reasoning is related to problem‐solving skills within addition and to reasoning skills outside addition. Ninety‐two 5‐ to 8‐year‐olds were asked to solve a series of conceptually related and unrelated addition problems, and the speed and accuracy of all self‐reported strategies were used to monitor their addition performance. Children were also given a series of general relational reasoning tasks to assess their ability to solve problems based on thematic, causal and visual relations. The results revealed that, while children were able to reason about commutativity relations, recognition of relations based on additive composition was rare. Furthermore, children's ability to reason about addition concepts increased with age and problem‐solving proficiency. Reasoning about addition concepts was related to performance on the thematic, causal and visual reasoning tasks for older children but not for younger children. Overall, the findings suggest that while children's early knowledge of addition relations is domain specific, as children develop in their broader reasoning abilities these developments enhance their addition reasoning.

Chemotherapy and irradiation in childhood Hodgkin's disease.

Eighty children aged less than 16 years with newly diagnosed Hodgkin's disease were treated between 1974 and 1982. Complete remission occurred in 95%, with actuarial five year overall survival of 94%, and relapse free survival of 82%: median follow up was 4.8 years. Sixty one children were staged clinically while 19 had staging laparotomies before treatment. Most received combined modality treatment with Ch1VPP chemotherapy (chlorambucil, vinblastine, procarbazine, and prednisolone) followed by irradiation of initial bulk disease. Nodular sclerosis predominated in both sexes, accounting for 60% of the total. Girls with stage IV disease, nodal sclerosis histology, and bulky mediastinal masses had a relatively poor prognosis. Ten children have relapsed, and three prolonged (6 to 7 years) second remissions have been observed. Four died of disease, and one from infection. Clinical staging, avoiding splenectomy, reduced the risk of serious infections. Our current policy is to treat stage IA disease with local irradiation and all other stages with chemotherapy, adding irradiation for bulky mediastinal disease.