Real-world Objects Research Articles

A Local Adversarial Attack with a Maximum Aggregated Region Sparseness Strategy for 3D Objects

The increasing reliance on deep neural network-based object detection models in various applications has raised significant security concerns due to their vulnerability to adversarial attacks. In physical 3D environments, existing adversarial attacks that target object detection (3D-AE) face significant challenges. These attacks often require large and dispersed modifications to objects, making them easily noticeable and reducing their effectiveness in real-world scenarios. To maximize the attack effectiveness, large and dispersed attack camouflages are often employed, which makes the camouflages overly conspicuous and reduces their visual stealth. The core issue is how to use minimal and concentrated camouflage to maximize the attack effect. Addressing this, our research focuses on developing more subtle and efficient attack methods that can better evade detection in practical settings. Based on these principles, this paper proposes a local 3D attack method driven by a Maximum Aggregated Region Sparseness (MARS) strategy. In simpler terms, our approach strategically concentrates the attack modifications to specific areas to enhance effectiveness while maintaining stealth. To maximize the aggregation of attack-camouflaged regions, an aggregation regularization term is designed to constrain the mask aggregation matrix based on the face-adjacency relationships. To minimize the attack camouflage regions, a sparseness regularization is designed to make the mask weights tend toward a U-shaped distribution and limit extreme values. Additionally, neural rendering is used to obtain gradient-propagating multi-angle augmented data and suppress the model’s detection to locate universal critical decision regions from multiple angles. These technical strategies ensure that the adversarial modifications remain effective across different viewpoints and conditions. We test the attack effectiveness of different region selection strategies. On the CARLA dataset, the average attack efficiency of attacking the YOLOv3 and v5 series networks reaches 1.724, which represents an improvement of 0.986 (134%) compared to baseline methods. These results demonstrate a significant enhancement in attack performance, highlighting the potential risks to real-world object detection systems. The experimental results demonstrate that our attack method achieves both stealth and aggressiveness from different viewpoints. Furthermore, we explore the transferability of the decision regions. The results indicate that our method can be effectively combined with different texture optimization methods, with the average precision decreasing by 0.488 and 0.662 across different networks, which indicates a strong attack effectiveness.

Particle swarm optimization with local search for height-map surface reconstruction from point clouds in reverse engineering

AbstractSurface reconstruction is a classical process in industrial engineering and manufacturing, particularly in reverse engineering, where the goal is to obtain a digital model from a physical object. For that purpose, the real object is typically scanned or digitized and the resulting point cloud is then fitted to mathematical surfaces through numerical optimization. The choice of the approximating functions is crucial for the accuracy of the process. Real-world objects such as manufactured workpieces often require complex nonlinear approximating functions, which are not well suited for standard numerical methods. In a previous paper presented at the ISMSI 2023 conference, we addressed this issue by using manually selected approximation functions via optimization through the cuckoo search algorithm with Lévy flights. Building upon that work, this paper presents an enhanced and extended method for surface reconstruction by using height-map surfaces obtained through a combination of exponential, polynomial and logarithmic functions. A feasible approach for this goal is to consider continuous bivariate distribution functions, which ensures consistent models along with good mathematical properties for the output shapes, such as smoothness and integrability. However, this approach leads to a difficult multivariate, constrained, multimodal continuous nonlinear optimization problem. To tackle this issue, we apply particle swarm optimization, a popular swarm intelligence technique for continuous optimization. The method is hybridized with a local search procedure for further improvement of the solutions and applied to a benchmark of 15 illustrative examples of point clouds fitted to different surface models. The performance of the method is analyzed through several computational experiments. The numerical and graphical results show that the method is able to recover the shape of the point clouds accurately and automatically. Furthermore, our approach outperforms other alternative methods significantly in terms of the numerical errors. We conclude that our method can be successfully applied to the reconstruction of manufactured workpieces in real industrial settings.

Visual information processing of 2D, virtual 3D and real-world objects marked by theta band responses: Visuospatial processing and cognitive load as a function of modality.

While pictures share global similarities with the real-world objects they depict, the latter have unique characteristics going beyond 2D representations. Due to its three-dimensional presentation mode, Virtual Reality (VR) is increasingly used to further approach real-world visual processing, yet it remains unresolved to what extent VR yields process comparable to real-world processes. Consequently, our study examined visuospatial processing by a triangular comparison of 2D objects, virtual 3D objects and real 3D objects. The theta band response (TBR) was analysed as an electrophysiological correlate of visual processing, allowing for the differentiation of predominantly stimulus-driven processes mirrored in the evoked response and internal, complex processing reflected in the induced response. Our results indicate that the differences between conditions driven by sensory features go beyond a binary division into 2D and 3D materials but are based on further sensory features: The evoked posterior TBR differentiated between all conditions but revealed fewer differences between processing of real-world and VR objects. Moreover, the induced midfrontal TBR indicated higher cognitive load for 2D objects compared to VR and real-world objects, while no difference between both latter conditions was revealed. In conclusion, our results demonstrate that the transferability of 2D- and VR-based findings to real-world processes depends to some degree on whether predominantly sensory stimulus features or higher cognitive processes are examined. Yet although VR and real-world processes are not to be equated based on our results, their comparison yielded fewer significant differences relative to the PC condition, advising the use of VR to examine visuospatial processing.

Competing working memory contents: perceptual over semantic prioritization and voluntary retrieval following retro-cueing

ABSTRACT This study investigated how voluntary and involuntary retrospective attention prioritize working memory (WM) representations of low-level visual features (perceptual dimension) and high-level categories (semantic dimension) using real-world objects. Reaction time and accuracy from two retro-cueing experiments were analyzed with a hierarchical drift-diffusion model to assess impacts on representation quality (drift rate) and retrieval time (non-decision time). Voluntariness of attention did not differentially affect perceptual and semantic WM contents. Drift rates showed stronger retro-cueing effects on perceptual compared to semantic contents, while non-decision times revealed retro-cueing effects only for voluntary attention. These findings suggest: (1) voluntariness does not differentiate between perceptual and semantic contents competing in WM; (2) attention prioritizes perceptual over semantic contents; and (3) voluntariness is critical for retrieving WM contents in advance of decision-making. This work highlights how WM content type and attentional voluntariness independently shape the effects of retrospective attention.

Interfacing with history: curating with audio augmented objects

ABSTRACT This article presents and discusses the results from visitors’ interactions with two audio augmented reality experiences containing audio augmented objects; physical, real-world objects to which virtual audio sources have been attached. It then proceeds to discusses the commonly identified themes arising from the observation of visitors’ behaviour within these experiences and the analysis of their verbal and written feedback. The curatorial potential of audio augmented objects is discussed and, by way of conclusion, their functionality as interfaces to digital audio archival content is proposed, along with their ability to reframe, re-contextualise and create renewed experiences with existing collections of silenced museum exhibits.

Determination of the aerodynamic drag coefficient for objects with real and scaled dimensions on the example of two vehicle bodywork concepts

Abstract With the development of motorization and the increase in vehicle speed, determining aerodynamic drag has become a key element in optimizing car design. Reducing air drag has a direct impact on fuel consumption, performance, and vehicle safety. The aim of this work is to numerically verify the effect of the Reynolds number on the obtained values of the aerodynamic drag coefficient Cd for both real-world objects and their models in a scale of 1:10. An additional aim is to compare the obtained results of numerical tests for 2 exemplary concepts of racing car bodywork.

Foraging and inertia: understanding the developmental dynamics of overt visual attention.

During early life, we develop the ability to choose what we focus on and what we ignore, allowing us to regulate perception and action in complex environments. But how does this change influence how we spontaneously allocate attention to real-world objects during free behaviour? Here, in this narrative review, we examine this question by considering the time dynamics of spontaneous overt visual attention, and how these develop through early life. Even in early childhood, visual attention shifts occur both periodically and aperiodically. These reorientations become more internally controlled as development progresses. Increasingly with age, attention states also develop self-sustaining attractor dynamics, known as attention inertia, in which the longer an attention episode lasts, the more the likelihood increases of its continuing. These self-sustaining dynamics are driven by amplificatory interactions between engagement, comprehension, and distractibility. We consider why experimental measures show decline in sustained attention over time, while real-world visual attention often demonstrates the opposite pattern. Finally, we discuss multi-stable attention states, where both hypo-arousal (mind-wandering) and hyper-arousal (fragmentary attention) may also show self-sustaining attractor dynamics driven by moment-by-moment amplificatory child-environment interactions; and we consider possible applications of this work, and future directions.

Prediction-based false memory: Unconfirmed prediction can result in robust false memories

A growing body of literature suggests a powerful role of predictions on memory through prediction violation and prediction confirmation. Violation appears to enhance memory for the event violating the prediction, meanwhile, confirmation boosts memory for the predicted event instead. Crucially, however, the effect of prediction by itself has not been identified as it has typically been studied with its violation or confirmation. Here, we demonstrate the power of explicit predictions on memory by isolating it from its direct violation and confirmation. In a series of experiments, participants were presented with a real-world object along with three characters and they predicted which character the object belonged to. Upon prediction, participants received either visual confirmation (predicted character showing the item), visual rebuttal (another character showing the item) or no feedback (none of the characters showing the item) with regard to their prediction. When their memory was tested, participants were more likely to falsely remember that their predicted character showed them the item than the other characters did, even when no feedback was provided. This false memory was not eliminated by visual rebuttal and it was not weakened when participants had a strong item memory. Experiments 2–4 eliminated action (selecting a predicted character) as an alternative explanation and demonstrated that this prediction-based false memory could be modulated through indirect prediction confirmation and rebuttal. Taken together, our findings show that explicit predictions can be sufficient to induce false memory of predicted events that are robust enough to withstand its direct rebuttal.

Classification of Real-World Objects Using Supervised ML-Assisted Polarimetry: Cost/Benefit Analysis

Classification of Real-World Objects Using Supervised ML-Assisted Polarimetry: Cost/Benefit Analysis

Multilingual entity alignment by abductive knowledge reasoning on multiple knowledge graphs

Objectives:Entity alignment (EA) seeks to identify similar real-world objects in different multilingual knowledge graphs (KGs), also known as ontology alignment. EA assists in handling a wide range of language semantics and in building integrated knowledge bases. However, most mainstream studies have focused on structural information, paying little attention to insufficient contextual information and limited handling of complex relationships. This paper aims to address these limitations and improve EA performance and efficiency. Methods:This paper investigates multilingual EA techniques and proposes a novel Abductive Knowledge Reasoning (AKR) model to address these issues. AKR can compute complex relationship semantics context by reasoning and enrich counterpart entity contextual information through centrality calculation, which helps connect distant entities in multilingual KGs. Novelty:The proposed AKR model introduces a new approach to EA by integrating centrality calculation and relational semantics reasoning. This method overcomes the limitations of existing EA techniques by effectively handling insufficient contextual information and complex relationships in multilingual KGs. Findings:AKR outperforms all state-of-the-art EA models across five datasets. AKR achieves Hit@1 score of 79.4%, for entity alignment between Chinese-to-English knowledge graphs representing 19.9% improvement over the best-performing translation-based model, Neighborhood-Aware Attentional Representation Entity Alignment, and a 5.0% improvement over the best-performing graph neural network-based model, Relational Semantics Augmentation.

MVImgNet2.0: A Larger-scale Dataset of Multi-view Images

MVImgNet is a large-scale dataset that contains multi-view images of ~220k real-world objects in 238 classes. As a counterpart of ImageNet, it introduces 3D visual signals via multi-view shooting, making a soft bridge between 2D and 3D vision. This paper constructs the MVImgNet2.0 dataset that expands MVImgNet into a total of ~520k objects and 515 categories, which derives a 3D dataset with a larger scale that is more comparable to ones in the 2D domain. In addition to the expanded dataset scale and category range, MVImgNet2.0 is of a higher quality than MVImgNet owing to four new features: (i) most shoots capture 360° views of the objects, which can support the learning of object reconstruction with completeness; (ii) the segmentation manner is advanced to produce foreground object masks of higher accuracy; (iii) a more powerful structure-from-motion method is adopted to derive the camera pose for each frame of a lower estimation error; (iv) higher-quality dense point clouds are reconstructed via advanced methods for objects captured in 360 ° views, which can serve for downstream applications. Extensive experiments confirm the value of the proposed MVImgNet2.0 in boosting the performance of large 3D reconstruction models. MVImgNet2.0 will be public at luyues.github.io/mvimgnet2 , including multi-view images of all 520k objects, the reconstructed high-quality point clouds, and data annotation codes, hoping to inspire the broader vision community.

A new outlier rejection approach for non-Lambertian photometric stereo

Photometric stereo (PS) has garnered increasing attention due to its adeptness in restoring local fine textures. However, non-Lambertian reflections present in almost all real-world objects limit the effectiveness of the Lambertian model for surface normal vector estimation. Although BRDF-based and deep learning-based methods have become mainstream, recent research shows that comparable accuracy can also be achieved through simple filtering of observed intensity values. Nevertheless, these methods only consider the relative bias of pixel values and require manual specification of the number of pixels to be culled and the number of iterations. To address these issues, this paper proposes corresponding improvement methods. Firstly, a weighted inter-relationship function (IRF) fused with Huber loss is introduced to robustly and effectively evaluate the abnormal degree of pixel value. Secondly, based on the IRF curve and histogram statistical analysis, the number of excluded pixels is adaptively calculated. Thirdly, linear equations are then constructed based on the photometric equations, and the maximum between-class variance method is employed to achieve a high degree of sparsity, enabling fast and effective normal vector estimation. Finally, to verify the effectiveness of the proposed algorithm for non-Lambertian PS vision, quantitative verification tests on the synthetic dataset and the open-source datasets “DiLiGenT” and “DiLiGenT-PI” in real-world scenarios, and qualitative assessment experiments on real metal roughness samples are conducted. The experimental results demonstrate that, compared with the position threshold and IRF methods, our algorithms not only significantly enhance the accuracy of normal vector solutions but also markedly improve the operational efficiency of the algorithm, laying a solid foundation for practical online applications. These results fully validate the correctness and effectiveness of the proposed algorithm and provide a reference for the further development of outlier removal algorithms.

Discovering data spaces: A classification of design options

Technical coordination between organizations and security concerns are among the major barriers to data sharing. Data spaces are an emerging digital infrastructure that helps address these challenges by sovereignly sharing data across institutional boundaries. The data space concept is at the core of many high-profile research initiatives in the European Union and receives great adoption in practice. Despite the great interest, there is, however, a demand for more conceptual clarity and approaches to describe and design them purposefully. We propose a taxonomy of data space design options grounded in a literature review, an analysis of real-world objects, and over nine hours of expert interviews with data space initiatives. The taxonomy advances our understanding of data space designs and gives a framework to practice making informed design decisions. Our work provides a comprehensive solution space for data space designers to (a) (re-)design data spaces more efficiently and (b) acquire a ‘big picture’ of what needs to be considered.

Evaluation of Cluster Algorithms for Radar-Based Object Recognition in Autonomous and Assisted Driving.

Perception systems for assisted driving and autonomy enable the identification and classification of objects through a concentration of sensors installed in vehicles, including Radio Detection and Ranging (RADAR), camera, Light Detection and Ranging (LIDAR), ultrasound, and HD maps. These sensors ensure a reliable and robust navigation system. Radar, in particular, operates with electromagnetic waves and remains effective under a variety of weather conditions. It uses point cloud technology to map the objects in front of you, making it easy to group these points to associate them with real-world objects. Numerous clustering algorithms have been developed and can be integrated into radar systems to identify, investigate, and track objects. In this study, we evaluate several clustering algorithms to determine their suitability for application in automotive radar systems. Our analysis covered a variety of current methods, the mathematical process of these methods, and presented a comparison table between these algorithms, including Hierarchical Clustering, Affinity Propagation Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Mini-Batch K-Means, K-Means Mean Shift, OPTICS, Spectral Clustering, and Gaussian Mixture. We have found that K-Means, Mean Shift, and DBSCAN are particularly suitable for these applications, based on performance indicators that assess suitability and efficiency. However, DBSCAN shows better performance compared to others. Furthermore, our findings highlight that the choice of radar significantly impacts the effectiveness of these object recognition methods.

Emotive Narrative: Thinking and Writing Experimental Animation through Things’ Perspective

The purpose of this study is to explore the magical nature of things, to investigate nonhuman entities, and to study from the philosophy of Object-Oriented Ontology. Rethinking Paul Wells’ scripted artefact based on the ontography concept of Alien Phenomenology, specifically for the transition of objects from real world objects to cinematic objects, Animators go through the found or made object and object as narrative stages. To explore the potential of things, we engage creators in co-speculation through Evocative Object Cards activity that evokes experience. By thinking about the theme of the secret life of things and experimenting with writing from thing’s perspective, we can realize the emotive narrative of things. Using the concept of “human-object” experience entanglement, this study conducted an experimental animation writing workshop to stimulate the tactile memory of the creators, so that they can understand how objects reveal their unique agency in the animation, and then understand the satisfaction of the participants through semi-structured qualitative interviews. The research results show that this activity has a positive impact on the creators’ creative writing of animation scripts, which not only puts forward alternative views beyond the mainstream animation stories, but also enriches the audience’s viewing experience.

L2T-DFM: Learning to Teach with Dynamic Fused Metric

The loss function plays a crucial role in the construction of machine learning algorithms. Employing a teacher model to set loss functions dynamically for student models has attracted attention. In existing works, (1) the characterization of the dynamic loss suffers from some inherent limitations, ie, the computational cost of loss networks and the restricted similarity measurement handcrafted loss functions; and (2) the states of the student model are provided to the teacher model directly without integration, causing the teacher model to underperform when trained on insufficient amounts of data. To alleviate the above-mentioned issues, in this paper, we select and weigh a set of similarity metrics by a confidence-based selection algorithm and a temporal teacher model to enhance the dynamic loss functions. Subsequently, to integrate the states of the student model, we employ statistics to quantify the information loss of the student model. Extensive experiments demonstrate that our approach can enhance student learning and improve the performance of various deep models on real-world tasks, including classification, object detection, and semantic segmentation scenarios.

Fast Broad Multiview Multi-Instance Multilabel Learning (FBM3L) With Viewwise Intercorrelation.

Multiview multi-instance multilabel learning (M3L) is a popular research topic during the past few years in modeling complex real-world objects such as medical images and subtitled video. However, existing M3L methods suffer from relatively low accuracy and training efficiency for large datasets due to several issues: 1) the viewwise intercorrelation (i.e., the correlations of instances and/or bags between different views) are neglected; 2) the diverse correlations (e.g., viewwise intercorrelation, interinstance correlation, and interlabel correlation) are not jointly considered; and 3) high computation burden for training process over bags, instances, and labels across different views. To resolve these issues, a novel framework called fast broad M3L (FBM3L) is proposed with three innovations: 1) utilization of viewwise intercorrelation for better modeling of M3L tasks while existing M3L methods have not considered; 2) based on graph convolutional network (GCN) and broad learning system (BLS), a viewwise subnetwork is newly designed to achieve joint learning among the diverse correlations; and 3) under BLS platform, FBM3L can learn multiple subnetworks jointly across all views with significantly less training time. Experiments show that FBM3L is highly competitive (or even better than) in all evaluation metrics up to 64% in average precision (AP) and much faster than most M3L (or MIML) methods (up to 1030 times), especially on large multiview datasets ( ≥ 260 K objects).

What do we see behind an occluder? Amodal completion of statistical properties in complex objects.

When a spiky object is occluded, we expect its spiky features tocontinue behind the occluder. Although many real-world objects contain complex features, it is unclear how more complex features are amodally completed and whether this process is automatic. To investigate this issue, we created pairs of displays with identical contour edges up to the point of occlusion, but with occluded portions exchanged. We then asked participants to search for oddball targets among distractors and asked whether relations between searches involving occluded displays would match better with relations between searches involving completions that are either globally consistent or inconsistent with the visible portions of these displays. Across two experiments involving simple and complex shapes, search times involving occluded displays matched better with those involving globally consistent compared with inconsistent displays. Analogous analyses on deep networks pretrained for object categorization revealed a similar pattern of results for simple but not complex shapes. Thus, deep networks seem to extrapolate simple occluded contours but not more complex contours. Taken together, our results show that amodal completion in humans is sophisticated and can be based on extrapolating global statistical properties.

A larger augmented-reality field of view improves interaction with nearby holographic objects.

Augmented-reality (AR) applications have shown potential for assisting and modulating gait in health-related fields, like AR cueing of foot-placement locations in people with Parkinson's disease. However, the size of the AR field of view (AR-FOV), which is smaller than one's own FOV, might affect interaction with nearby floor-based holographic objects. The study's primary objective was to evaluate the effect of AR-FOV size on the required head orientations for viewing and interacting with real-world and holographic floor-based objects during standstill and walking conditions. Secondary, we evaluated the effect of AR-FOV size on gait speed when interacting with real-world and holographic objects. Sixteen healthy middle-aged adults participated in two experiments wearing HoloLens 1 and 2 AR headsets that differ in AR-FOV size. To confirm participants' perceived differences in AR-FOV size, we examined the head orientations required for viewing nearby and far objects from a standstill position (Experiment 1). In Experiment 2, we examined the effect of AR-FOV size on head orientations and gait speeds for negotiating 2D and 3D objects during walking. Less downward head orientation was required for looking at nearby holographic objects with HoloLens 2 than with HoloLens 1, as expected given differences in perceived AR-FOV size (Experiment 1). In Experiment 2, a greater downward head orientation was observed for interacting with holographic objects compared to real-world objects, but again less so for HoloLens 2 than HoloLens 1 along the line of progression. Participants walked slightly but significantly slower when interacting with holographic objects compared to real-world objects, without any differences between the HoloLenses. To conclude, the increased size of the AR-FOV did not affect gait speed, but resulted in more real-world-like head orientations for seeing and picking up task-relevant information when interacting with floor-based holographic objects, improving the potential efficacy of AR cueing applications.

High-Visibility Edge-Highlighting Visualization of 3D Scanned Point Clouds Based on Dual 3D Edge Extraction

Recent advances in 3D scanning have enabled the digital recording of complex objects as large-scale point clouds, which require clear visualization to convey their 3D shapes effectively. Edge-highlighting visualization is used to improve the comprehensibility of complex 3D structures by enhancing the 3D edges and high-curvature regions of the scanned objects. However, traditional methods often struggle with real-world objects due to inadequate representation of soft edges (i.e., rounded edges) and excessive line clutter, impairing resolution and depth perception. To address these challenges, we propose a novel visualization method for 3D scanned point clouds based on dual 3D edge extraction and opacity–color gradation. Dual 3D edge extraction separately identifies sharp and soft edges, integrating both into the visualization. Opacity–color gradation enhances the clarity of fine structures within soft edges through variations in color and opacity, while also creating a halo effect that improves both resolution and depth perception of the visualized edges. Computation times required for dual 3D edge extraction are comparable to conventional binary statistical edge-extraction methods. Visualizations with opacity–color gradation are executable at interactive rendering speeds. The effectiveness of the proposed method is demonstrated using 3D scanned point cloud data from high-value cultural heritage objects.