Images Of Real-world Objects Research Articles

One-Shot Learning from Prototype Stock Keeping Unit Images

This paper highlights the importance of one-shot learning from prototype Stock Keeping Unit (SKU) images for efficient product recognition in retail and inventory management. Traditional methods require large supervised datasets to train deep neural networks, which can be costly and impractical. One-shot learning techniques mitigate this issue by enabling classification from a single prototype image per product class, thus reducing data annotation efforts. We introduce the Variational Prototyping Encoder (VPE), a novel deep neural network for one-shot classification. Utilizing a support set of prototype SKU images, VPE learns to classify query images by capturing image similarity and prototypical concepts. Unlike metric learning-based approaches, VPE pre-learns image translation from real-world object images to prototype images as a meta-task, facilitating efficient one-shot classification with minimal supervision. Our research demonstrates that VPE effectively reduces the need for extensive datasets by utilizing a single image per class while accurately classifying query images into their respective categories, thus providing a practical solution for product classification tasks.

Deceiving Humans and Machines Alike: Search-based Test Input Generation for DNNs Using Variational Autoencoders

Due to the rapid adoption of Deep Neural Networks (DNNs) into larger software systems, testing of DNN-based systems has received much attention recently. While many different test adequacy criteria have been suggested, we lack effective test input generation techniques. Inputs such as images of real-world objects and scenes are not only expensive to collect but also difficult to randomly sample. Consequently, current testing techniques for DNNs tend to apply small local perturbations to existing inputs to generate new inputs. We propose SINVAD (Search-based Input space Navigation using Variational AutoencoDers), a way to sample from, and navigate over, a space of realistic inputs that resembles the true distribution in the training data. Our input space is constructed using Variational Autoencoders (VAEs), and navigated through their latent vector space. Our analysis shows that the VAE-based input space is well-aligned with human perception of what constitutes realistic inputs. Further, we show that this space can be effectively searched to achieve various testing scenarios, such as boundary testing of two different DNNs or analyzing class labels that are difficult for the given DNN to distinguish. Guidelines on how to design VAE architectures are presented as well. Our results have the potential to open the field to meaningful exploration through the space of highly structured images.

Benchmarking and Analysis of Unsupervised Object Segmentation from Real-World Single Images

In this paper, we study the problem of unsupervised object segmentation from single images. We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-world images. We first introduce seven complexity factors to quantitatively measure the distributions of background and foreground object biases in appearance and geometry for datasets with human annotations. With the aid of these factors, we empirically find that, not surprisingly, existing unsupervised models fail to segment generic objects in real-world images, although they can easily achieve excellent performance on numerous simple synthetic datasets, due to the vast gap in objectness biases between synthetic and real images. By conducting extensive experiments on multiple groups of ablated real-world datasets, we ultimately find that the key factors underlying the failure of existing unsupervised models on real-world images are the challenging distributions of background and foreground object biases in appearance and geometry. Because of this, the inductive biases introduced in existing unsupervised models can hardly capture the diverse object distributions. Our research results suggest that future work should exploit more explicit objectness biases in the network design.

Information system for monitoring number of fruits and their condition using computer vision technology

The main objectives of process automation are to improve the quality and productivity of process execution. An automated process has more stable characteristics than a manually executed process. One of the tools of automation is artificial intelligence in general and computer vision in particular. Computer vision (also called machine vision) is a scientific trend in the field of artificial intelligence and related technologies for acquiring images of real-world objects, processing them and using the obtained data to solve various kinds of applied tasks without (full or partial) human participation. From an engineering perspective, computer vision seeks to automate tasks that can be performed by the human visual system. Computer vision tasks include ways of acquiring, processing, analysing, and understanding digital images and extracting data from the real world to produce numerical or symbolic information. In the context of crop farming, automation is primarily capable of increasing yields and improving the quality of the final product, by better controlling environmental conditions and timely detection of any abnormalities in the development and growth of plants and fruits, by increasing productivity, by replacing a variety of more accurate and responsive automated systems (e.g. irrigation or temperature control) with human involvement, and thereby simplifying the role of humans in cultivation, and by improving the quality of the final product. In the paper, the main objective of the designed information system is the recognition in a digital image of plant fruits with subsequent classification based on known colour ranges in advance. In order to achieve this goal, the following tasks should be solved: minimisation of possible negative effects of distortions of the original image; analysing the image to look for tomato fruits on it.

Hippocampal Functions Modulate Transfer-Appropriate Cortical Representations Supporting Subsequent Memory.

The hippocampus plays a central role as a coordinate system or index of information stored in neocortical loci. Nonetheless, it remains unclear how hippocampal processes integrate with cortical information to facilitate successful memory encoding. Thus, the goal of the current study was to identify specific hippocampal-cortical interactions that support object encoding. We collected fMRI data while 19 human participants (7 female and 12 male) encoded images of real-world objects and tested their memory for object concepts and image exemplars (i.e., conceptual and perceptual memory). Representational similarity analysis revealed robust representations of visual and semantic information in canonical visual (e.g., occipital cortex) and semantic (e.g., angular gyrus) regions in the cortex, but not in the hippocampus. Critically, hippocampal functions modulated the mnemonic impact of cortical representations that are most pertinent to future memory demands, or transfer-appropriate representations Subsequent perceptual memory was best predicted by the strength of visual representations in ventromedial occipital cortex in coordination with hippocampal activity and pattern information during encoding. In parallel, subsequent conceptual memory was best predicted by the strength of semantic representations in left inferior frontal gyrus and angular gyrus in coordination with either hippocampal activity or semantic representational strength during encoding. We found no evidence for transfer-incongruent hippocampal-cortical interactions supporting subsequent memory (i.e., no hippocampal interactions with cortical visual/semantic representations supported conceptual/perceptual memory). Collectively, these results suggest that diverse hippocampal functions flexibly modulate cortical representations of object properties to satisfy distinct future memory demands.Significance Statement The hippocampus is theorized to index pieces of information stored throughout the cortex to support episodic memory. Yet how hippocampal processes integrate with cortical representation of stimulus information remains unclear. Using fMRI, we examined various forms of hippocampal-cortical interactions during object encoding in relation to subsequent performance on conceptual and perceptual memory tests. Our results revealed novel hippocampal-cortical interactions that utilize semantic and visual representations in transfer-appropriate manners: conceptual memory supported by hippocampal modulation of frontoparietal semantic representations, and perceptual memory supported by hippocampal modulation of occipital visual representations. These findings provide important insights into the neural mechanisms underlying the formation of information-rich episodic memory and underscore the value of studying the flexible interplay between brain regions for complex cognition.

Keep it real, keep it simple: the effects of icon characteristics on visual search

ABSTRACT Previous research examining how icons’ concreteness, visual complexity, and distinctiveness influence visual search performance have led to disagreements over which icon characteristic most affects behaviour. These icon characteristics are often poorly defined and interrelated, particularly concreteness. Accordingly, drawing strong inferences about the robustness of concreteness as a factor in search for visual icons is challenging. Here, we operationalised concreteness into three distinct levels: concrete icons were images of real-world objects, photorealistic icons were drawings of the object, and abstract icons were images with no conceptual information. Across two experiments, participants rated each icon on various icon characteristics (e.g. concreteness, visual complexity) to provide a ground truth for these factors and to validate our concreteness manipulation. In a separate study, naive participants performed a visual search task for a target icon. Oculomotor measures were utilised to elucidate how various icon characteristics affected search performance. Although we were unable to fully disassociate concreteness from visual complexity, we found that icons high in concreteness improved search performance, but as visual complexity increased, object identification became slower. This was largely demonstrated through increased verification times for complex targets. The present set of studies indicate that highly concrete and simple icons engender search benefits.

Semantic segmentation using Firefly Algorithm-based evolving ensemble deep neural networks

Automatic segmentation of salient objects in real-world images has gained increasing interests owing to its popularity in diverse real-world applications, such as autonomous driving, medical diagnosis, aviation security, and underwater surveillance. In this research, we propose Firefly Algorithm (FA)-enhanced evolving ensemble deep networks for semantic segmentation and visual saliency prediction. An improved FA model is proposed to optimize network hyper-parameters. Specifically, it employs mutation operators and a neighbouring search strategy with granular search steps to establish search intensification. It also emphasizes search diversification by adopting multiple dynamic hybrid leaders and diverse adaptive sine and cosine search trajectories in full and randomly selected sub-dimensions to overcome stagnation. Because of its competent segmentation performance, DeepLabV3+ is fine-tuned using transfer learning with FA-based hyper-parameter identification. We optimize the learning rate, momentum and weight decay of the transfer learning network. A number of optimized DeepLabV3+ networks with distinguishing learning configurations are yielded. An ensemble model is subsequently constructed by incorporating three optimized base networks to further strengthen segmentation performance. Evaluated using diverse challenging semantic segmentation and saliency prediction tasks using underwater and medical image data sets, our evolving ensemble deep network illustrates significant superiority over other state-of-the-art deep networks and existing studies. The proposed FA model also outperforms other search methods in solving diverse mathematical landscapes with statistical significance.

Objects’ perceived meaningfulness predicts both subjective memorability judgments and actual memory performance

ABSTRACT Memorability studies have revealed a limitation in our ability to accurately judge which images are memorable. Conversely, metacognitive research suggests that individuals can utilize cues to reliably assess their memory performance. Here, we investigated two important stimulus cues potentially underlying subjective memorability, and their relation to actual memory performance. Participants encoded 200 real-world object images while providing Judgements of Learning (JOLs), in which they estimated the likelihood of remembering each image. Subsequently, they completed an old/new memory recognition test on these stimuli. All stimuli were priorly rated by an independent group of participants for their perceived meaningfulness and visual complexity. Results indicated that participants' metacognitive judgements exhibited a relatively good resolution, allowing them to distinguish well-remembered from less-remembered images. Furthermore, analyses conducted at the image level demonstrated that JOLs significantly predicted memory performance, with the meaningfulness of the images emerging as a crucial factor associated with JOLs of both participants and their actual memory performance. The visual complexity factor was correlated only with the actual memory performance. Collectively, these findings suggest that memory performance is closely associated with the meaning of objects. Critically, individuals possess a subjective sense of images' memorability, which is at least partially mediated by perceived meaningfulness.

Freezing Sets Invariant-based Characteristics of Digital Images

Due to the widespread use of digital images of real-world objects as mathematical models, this research examines the freezing sets invariant-base properties of digital images. In contrast to earlier studies that only covered a discrete or limited collection of points, fixed points of digitally continuous functions are approved to deal with a variety of characteristics of digital images.

Dissociating the Impact of Memorability on Electrophysiological Correlates of Memory Encoding Success.

Despite its unlimited capacity, not all visual information we encounter is encoded into visual long-term memory. Traditionally, variability in encoding success has been ascribed to variability in the types and efficacy of an individual's cognitive processes during encoding. Accordingly, past studies have identified several neural correlates of variability in encoding success, namely, frontal positivity, occipital alpha amplitude, and frontal theta amplitude, by contrasting the electrophysiological signals recorded during successful and failed encoding processes (i.e., subsequent memory). However, recent research demonstrated individuals remember and forget consistent sets of stimuli, thereby elucidating stimulus-intrinsic factors (i.e., memorability) that determine the ease of memory encoding independent of individual-specific variability in encoding processes. The existence of memorability raises the possibility that canonical EEG correlates of subsequent memory may reflect variability in stimulus-intrinsic factors rather than individual-specific encoding processes. To test this, we recorded the EEG correlates of subsequent memory while participants encoded 600 images of real-world objects and assessed the unique contribution of individual-specific and stimulus-intrinsic factors on each EEG correlate. Here, we found that frontal theta amplitude and occipital alpha amplitude were only influenced by individual-specific encoding success, whereas frontal positivity was influenced by stimulus-intrinsic and individual-specific encoding success. Overall, our results offer novel interpretations of canonical EEG correlates of subsequent memory by demonstrating a dissociable impact of stimulus-intrinsic and individual-specific factors of memory encoding success.

Macaques recognize features in synthetic images derived from ventral stream neurons

Primates can recognize features in virtually all types of images, an ability that still requires a comprehensive computational explanation. One hypothesis is that visual cortex neurons learn patterns from scenes, objects, and textures, and use these patterns to interpolate incoming visual information. We have used machine learning algorithms to instantiate visual patterns stored by neurons-we call these highly activating images prototypes. Prototypes from inferotemporal (IT) neurons often resemble parts of real-world objects, such as monkey faces and body parts, a similarity established via pretrained neural networks [C. R. Ponce et al.,Cell 177, 999-1009.e10 (2019)]and naïve human participants [A. Bardon, W. Xiao, C. R. Ponce, M. S. Livingstone, G. Kreiman, Proc. Natl. Acad. Sci. U.S.A. 119, e2118705119 (2022)]. However, it is not known whether monkeys themselves perceive similarities between neuronal prototypes and real-world objects. Here, we investigated whether monkeys reported similarities between prototypes and real-world objects using a two-alternative forced choice task. We trained the animals to saccade to synthetic images of monkeys, and subsequently tested how they classified prototypes synthesized from IT and primary visual cortex (V1). We found monkeys classified IT prototypes as conspecifics more often than they did random generator images and V1 prototypes, and their choices were partially predicted by convolutional neural networks. Further, we confirmed that monkeys could abstract general shape information from images of real-world objects. Finally, we verified these results with human participants. Our results provide further evidence that prototypes from cortical neurons represent interpretable abstractions from the visual world.

Variational Multi-Prototype Encoder for Object Recognition Using Multiple Prototype Images

In the recent research of Variational Prototyping-Encoder (VPE), the problem of classifying 2D flat objects of the unseen class has been addressed. VPE solves this problem by pre-learning the image translation task from real-world object images to their corresponding prototype images as a meta-task. VPE uses a single prototype for each object class. However, in general, a single prototype is not sufficient to represent a generic object class because the appearance can change significantly according to viewpoints and other factors. In this case, using VPE and a single prototype for each class in training can result in overfitting or performance degradation. One solution may be the use of multiple prototypes. However, this also requires costly sub-labeling for dividing the input class into smaller classes and assigning a prototype to each. Therefore, we propose a new learning method, the variational multi-prototype encoder (VaMPE), which can overcome the limitations of VPE and use multiple prototypes for each object class. The proposed method does not require additional sub-labeling other than simply adding multiple prototypes to each class. Through various experiments, we demonstrate that the proposed method outperforms VPE.

Semantic influence on visual working memory of object identity and location

Does semantic information—in particular, regularities in category membership across objects—influence visual working memory (VWM) processing? We predict that the answer is “yes”. Four experiments evaluating this prediction are reported. Experimental stimuli were images of real-world objects arranged in either one or two spatial clusters. On coherent trials, all objects belonging to a cluster also belonged to the same category. On incoherent trials, at least one cluster contained objects from different categories. Experiments using a change-detection paradigm (Experiments 1–3) and an experiment in which participants recalled the locations of objects in a scene (Experiment 4) yielded the same result: participants showed better memory performance on coherent trials than on incoherent trials. Taken as a whole, these experiments provide the best (perhaps only) data to date demonstrating that statistical regularities in semantic category membership improve VWM performance. Because a conventional perspective in cognitive science regards VWM as being sensitive solely to bottom-up visual properties of objects (e.g., shape, color, orientation), our results indicate that cognitive science may need to modify its conceptualization of VWM so that it is closer to “conceptual short-term memory”, a short-term memory store representing current stimuli and their associated concepts (Potter, 1993, 2012).

Combining convolutional neural networks and cognitive models to predict novel object recognition in humans.

Object representations from convolutional neural network (CNN) models of computer vision (LeCun, Bengio, & Hinton, 2015) were used to drive a cognitive model of decision making, the linear ballistic accumulator (LBA) model (Brown & Heathcote, 2008), to predict errors and response times (RTs) in a novel object recognition task in humans. CNNs have become very successful at visual tasks like classifying objects in real-world images (e.g., He, Zhang, Ren, & Sun, 2015; Krizhevsky, Sutskever, & Hinton, 2012). We asked whether object representations learned by CNNs previously trained on a large corpus of natural images could be used to predict performance recognizing novel objects the network has never been trained on; we used novel Greebles, Ziggerins, and Sheinbugs that have been used in a number of previous object recognition studies. We specifically investigated whether a model combining high-level CNN representations of these novel objects could be used to drive an LBA model of decision making to account for errors and RTs in a same-different matching task (from Richler et al., 2019). Combining linearly transformed CNN object representations with the LBA provided reasonable accounts of performance not only on average, but at the individual-participant level and the item level as well. We frame the findings in the context of growing interest in using CNN models to understand visual object representations and the promise of using CNN representations to extend cognitive models to explain more complex aspects of human behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Application of morphological processing methods in computer stereo vision

Computer vision algorithms are important for many areas of human activity. In particular, the number of applications related to the need to process images of real-world objects with computerized tools and the subsequent use of descriptive information in a variety of interactive and automated decision-making systems is increased. An important tool for analyzing real-world scenes are approaches to the application of stereo vision algorithms. The important step of many stereo matching algorithms is a disparity map. Depending on the content of the observed scene, part of the values on the disparity map can be immediately attributed to background values on a certain basis, or form a "natural" background, which is characterized by loss of informative data due to unacceptable error of subsequent resultant distance values. The calculated disparity map of any algorithm may contain some shortcomings in the form of discontinuities of continuous information areas caused by the complexity of shooting conditions, the impact of noise of various natures, hardware imperfections, and so on. An approach to mitigating the undesirable influence of negative factors on the resulting disparity is the use of mathematical morphology operations to process disparity maps at the post-processing stage. This paper presents information technology for increasing the content of disparity maps based on the mathematical morphology methods. The technology is based on a combination of morphological operations of erosion and dilation, which eliminates the typical problems of discontinuities of monotone regions and erroneous values on disparity maps. The proposed approach allows reducing the impact of common problems that arise during the operation of stereo matching algorithms, as well as increase the overall informativeness of disparity maps for images of real objects in the absence of partial or complete initial data on the characteristics of the observed scene. The results of testing morphological operations with disparity maps for real objects allow us to conclude about the possibility of partial restoration of areas of disparity maps with gaps in continuous information areas, as well as to reduce the impact of random anomalous values on the overall content of the disparity maps.

Efficient collection and automatic annotation of real-world object images by taking advantage of post-diminished multiple visual markers

ABSTRACTTo collect a human-annotated dataset for training deep convolutional neural networks is a very time-consuming and laborious process. To reduce this burden, we previously proposed an automated annotation by placing one visual marker above the detection target object in the training phase. However, in this approach, occasionally the marker hides the object surface. To avoid this issue, we propose placing a pedestal with multiple markers at the bottom of the object. If we use multiple markers, the object can be annotated even when the object hides some of the markers. Besides that, the simple modification of placing the markers on the bottom allows the use of simple background masking to avoid the neural network learning the remaining markers in the training image as a feature of the object. Background masking can completely remove the markers during the training process. Experiments showed the proposed vision system using our automatic object annotation outperformed the vision system using manual annotation in terms of object detection, orientation estimation, and 2D position estimation while reducing the time required for dataset collection from 16.1 hours to 7.30 hours.

Joint attention facilitates observed gaze direction discrimination

Efficiently judging where someone else is looking is important for social interactions, allowing us a window into their mental state by establishing joint attention. Previous work has shown that judging the gaze direction of a non-foveally presented face is facilitated when the eyes of that face are directed towards the centre of the scene. This finding has been interpreted as an example of the human bias for misattributing observed ambiguous gaze signals as self-directed eye-contact. To test this interpretation against an alternative hypothesis that the facilitation is instead driven by the establishment of joint attention, we conducted two experiments in which we varied the participants' fixation location. In both experiments, we replicated the previous finding of facilitated gaze discrimination when the participants fixated centrally. However, this facilitation was abolished when participants fixated peripheral fixation crosses (Experiment 1) and reversed when participants fixated peripheral images of real-world objects (Experiment 2). Based on these data, we propose that the facilitation effect is consistent with the interpretation that gaze discrimination is facilitated when joint attention is established. This finding therefore extends previous work showing that engaging in joint attention facilitates a range of social cognitive processes.

Resolution-enhanced quantum imaging by centroid estimation of biphotons

The spatial resolution of an optical system is limited by diffraction. Various schemes have been proposed to achieve resolution enhancement by employing either a scanning source/detector configuration or a two-photon response of the object. Here, we experimentally demonstrate a full-field resolution-enhancing scheme, based on the centroid estimation of spatially quantum-correlated biphotons. Our standard-quantum-limited scheme is able to image a general non-fluorescing object, using low-energy and low-intensity infrared illumination (i.e., with <0.001 photon per pixel per frame at 710 nm), achieving 41% of the theoretically available resolution enhancement. Images of real-world objects are shown for visual comparison, in which the classically bound resolution is surpassed using our technically straightforward quantum-imaging scheme.

The Problem of Perimeter

In Euclidean geometry, perimeter is a defined and readily measured property of shapes. In the real world, and especially in digitised images of real world objects, it is less well defined and much less readily measured, in spite of the fact that practically all image analysis programmes report numeric values, and use them in shape characterisation.

Investigating Object Representations in the Macaque Dorsal Visual Stream Using Single-unit Recordings.

Previous studies have shown that neurons in parieto-frontal areas of the macaque brain can be highly selective for real-world objects, disparity-defined curved surfaces, and images of real-world objects (with and without disparity) in a similar manner as described in the ventral visual stream. In addition, parieto-frontal areas are believed to convert visual object information into appropriate motor outputs, such as the pre-shaping of the hand during grasping. To better characterize object selectivity in the cortical network involved in visuomotor transformations, we provide a battery of tests intended to analyze the visual object selectivity of neurons in parieto-frontal regions.