Visual Signatures Research Articles

Controlling the scatterplot shapes of 2D and 3D multidimensional projections

Multidimensional projections are effective techniques for depicting high-dimensional data. The point patterns created by such techniques, or a technique’s visual signature, depend — apart from the data themselves — on the technique design and its parameter settings. Controlling such visual signatures — something that only few projections allow — can bring additional freedom for generating insightful depictions of the data. We present a novel projection technique — ShaRP — that allows explicit control on such visual signatures in terms of shapes of similar-value point clusters (settable to rectangles, triangles, ellipses, and convex polygons) and the projection space (2D or 3D Euclidean or S2). We show that ShaRP scales computationally well with dimensionality and dataset size, provides its signature-control by a small set of parameters, allows trading off projection quality to signature enforcement, and can be used to generate decision maps to explore the behavior of trained machine-learning classifiers.

#BlackatCMO: Challenging charter schools through youth Instagram counterstories

As protests flared in 2020, Black students took to Instagram to voice their experiences at ‘no-excuses’ Charter Management Organizations (CMOs). Such schools have presented a discourse of high achievement and social justice. Yet, in the span of a few weeks, hundreds of posts on Instagram offered rarely heard counter-narratives of the experience of being a student of Color at such schools. This paper analyzes how social media posts combine online discourse and youth culture to provide insight into the racialized experience of schooling. We show how these posts created distinct visual signatures, co-opted the well-crafted narratives of CMOs, and took advantage of features like comments and hashtags to foster solidarity and authenticity. We argue this social movement challenges the legitimacy of CMOs and their authority to teach children of Color.

Radiomic markers of intracerebral hemorrhage expansion on non-contrast CT: independent validation and comparison with visual markers.

To devise and validate radiomic signatures of impending hematoma expansion (HE) based on admission non-contrast head computed tomography (CT) of patients with intracerebral hemorrhage (ICH). Utilizing a large multicentric clinical trial dataset of hypertensive patients with spontaneous supratentorial ICH, we developed signatures predictive of HE in a discovery cohort (n = 449) and confirmed their performance in an independent validation cohort (n = 448). In addition to n = 1,130 radiomic features, n = 6 clinical variables associated with HE, n = 8 previously defined visual markers of HE, the BAT score, and combinations thereof served as candidate variable sets for signatures. The area under the receiver operating characteristic curve (AUC) quantified signatures' performance. A signature combining select radiomic features and clinical variables attained the highest AUC (95% confidence interval) of 0.67 (0.61-0.72) and 0.64 (0.59-0.70) in the discovery and independent validation cohort, respectively, significantly outperforming the clinical (pdiscovery = 0.02, pvalidation = 0.01) and visual signature (pdiscovery = 0.03, pvalidation = 0.01) as well as the BAT score (pdiscovery < 0.001, pvalidation < 0.001). Adding visual markers to radiomic features failed to improve prediction performance. All signatures were significantly (p < 0.001) correlated with functional outcome at 3-months, underlining their prognostic relevance. Radiomic features of ICH on admission non-contrast head CT can predict impending HE with stable generalizability; and combining radiomic with clinical predictors yielded the highest predictive value. By enabling selective anti-expansion treatment of patients at elevated risk of HE in future clinical trials, the proposed markers may increase therapeutic efficacy, and ultimately improve outcomes.

Brown bear body patches are temporally stable and represent a unique individual visual signature

Patches of color may be used to communicate to conspecifics, mainly in species showing uniform coloration, and may (a) help individuals maintain visual contact, such as between mothers and their young; (b) function as signals of subordination or to frighten rivals; (c) warn conspecifics of approaching predators; and/or (d) signal reproductive condition, health, or genetic quality to potential mates. Intraspecific communication represents one of the major evolutionary forces responsible for the coloration of body parts, but the meaning of many of these signals is still unclear. One of the first steps to understanding whether fur marks have a role in social communication is to understand whether such body patches are stable over time (i.e., whether they represent a unique visual signature for every individual). During the period 1999–2021, we recorded yearly pictures of 7 female (mean no. of monitoring years per bear = 13.6, standard deviation [SD] = 4.6; range = 9–22 yr) and 6 male (mean no. of monitoring years per bear = 9.3, SD = 4.3; range = 5–15 yr) brown bears (Ursus arctos) in the Cantabrian Mountains (NW Spain). We show that body mark shapes are stable over time and, because of their uniqueness, might represent a distinctive signature of individuals. Brown bear body marks may act as multicomponent signals, where different features of a given mark may inform about different aspects of the bearer or act as back-ups. For example, a quality-signaling capacity does not preclude the same mark from being used in other functions at the same time, such as individual recognition. Noninvasive techniques helpful for identifying individuals have been developed for estimating population size, reproductive rates, and the survival of several carnivore species. Fur marks that are stable over time can thus be useful in field research (e.g., body marks that are persistent and do not vary over time are an important tool in longitudinal photographic capture–recapture studies).

Aïda Muluneh's Homebound: A Journey in Photography &amp; Addis Foto Fest: Nine Years Survey

Aïda Muluneh's Homebound: A Journey in Photography &amp; Addis Foto Fest: Nine Years Survey

Photorealistic three-dimensional visualization of fusion datasets: cinematic rendering of PET/CT.

Cinematic rendering (CR) is a method of photorealistic 3D visualization of volumetric imaging data. We applied this technique to fusion PET/CT data. Two recent PET/CT cases were selected, one each of prostate-specific membrane antigen (PSMA)-targeted 18F-DCFPyL, and somatostatin-receptor-targeted 68Ga-DOTATATE. Targeted radiotracers were selected in order to provide high-contrast images for this proof-of-principle study. Cinematic rendering was performed with an enhanced algorithm that incorporated internal lighting within the PET-avid organs and lesions to allow for a distinct visual signature. The use of internal lighting for PET data provided CR of fused PET/CT scans. The interpreting radiologist must make judicious use of presets and cut planes in order to ensure important findings are not missed. CR of PET/CT data provides a photorealistic means of visualizing complex fusion imaging datasets. Such visualizations may aid anatomic understanding for surgical or procedural applications, may improve teaching of trainees, and may allow improved communication with patients.

Hidden signatures of early fire at Evron Quarry (1.0 to 0.8 Mya)

Pyrotechnology is a key element of hominin evolution. The identification of fire in early hominin sites relies primarily on an initial visual assessment of artifacts' physical alterations, resulting in potential underestimation of the prevalence of fire in the archaeological record. Here, we used a suite of spectroscopic techniques to counter the absence of visual signatures for fire and demonstrate the presence of burnt fauna and lithics at the Lower Paleolithic (LP) open-air site of Evron Quarry (Israel), dated between 1.0 and 0.8 Mya and roughly contemporaneous to Gesher Benot Ya'aqov where early pyrotechnology has been documented. We propose reexamining finds from other LP sites lacking visual clues of pyrotechnology to yield a renewed perspective on the origin, evolution, and spatiotemporal dispersal of the relationship between early hominin behavior and fire use.

Validation of a Novel Stereo Vibrometry Technique for Spiderweb Signal Analysis

Simple SummarySpiders often use their webs as sensory mechanisms, obtaining from them such information as the location of prey, the presence of rival spiders, and the characteristics of potential mates. Examining how this information is transmitted through the web and received by spiders is a promising biological area of research that could provide insight into a spider’s world and lead to new technologies that leverage these discoveries. In this paper, we develop a novel noncontact technique using two video cameras that is capable of analyzing vibrational signals transmitted through spiderwebs and validate this technique against the current standard of laser Doppler vibrometry. By combining the principles of stereo vision and video vibrometry, we can automatically extract three-dimensional vibrational information at any point in the spiderweb across time, and study how these signals propagate through the web. We show that this technique produces results comparable to those of standard laser vibrometry.From courtship rituals, to prey identification, to displays of rivalry, a spider’s web vibrates with a symphony of information. Examining the modality of information being transmitted and how spiders interact with this information could lead to new understanding how spiders perceive the world around them through their webs, and new biological and engineering techniques that leverage this understanding. Spiders interact with their webs through a variety of body motions, including abdominal tremors, bounces, and limb jerks along threads of the web. These signals often create a large enough visual signature that the web vibrations can be analyzed using video vibrometry on high-speed video of the communication exchange. Using video vibrometry to examine these signals has numerous benefits over the conventional method of laser vibrometry, such as the ability to analyze three-dimensional vibrations and the ability to take measurements from anywhere in the web, including directly from the body of the spider itself. In this study, we developed a method of three-dimensional vibration analysis that combines video vibrometry with stereo vision, and verified this method against laser vibrometry on a black widow spiderweb that was experiencing rivalry signals from two female spiders.

Glitch epistemologies for computational cities

This intervention advances glitches as epistemological vectors for apprehending and engaging the significance of digitally-mediated spatialities that appear nonperformative against normative scripts of urban computational paradigms. Drawing on two strands of contemporary thinking about glitches as systemic design features of digital systems and as generative fissures within them, we mobilize a queer orientation that stays with the generative tensions of urban spatialities that present as idiosyncratic and as interrupting. We mobilize this epistemological approach through illustrative U.S. based examples of seemingly abandoned shared e-bikes, performatively ‘ugly’ homes, and wilful property dilapidation wrought through the registers of desire and aesthetics. In so doing, we show how glitch empistemologies render visible how the technocapitalist manufacturing of normative spatial desires for particular kinds of urban sociospatialities and aesthetic visual signatures are both secured and interrupted on digitally-mediated and -mediatized terrains. Glitch epistemologies establish the significance of small-scale disorientations in digital urban mediations, engaging these nonperformativities and non-computes as unexceptional openings onto everyday possibilities for politics in computational cities.

LPMP: A Bio-Inspired Model for Visual Localization in Challenging Environments.

Autonomous vehicles require precise and reliable self-localization to cope with dynamic environments. The field of visual place recognition (VPR) aims to solve this challenge by relying on the visual modality to recognize a place despite changes in the appearance of the perceived visual scene. In this paper, we propose to tackle the VPR problem following a neuro-cybernetic approach. To this end, the Log-Polar Max-Pi (LPMP) model is introduced. This bio-inspired neural network allows building a neural representation of the environment via an unsupervised one-shot learning. Inspired by the spatial cognition of mammals, visual information in the LPMP model are processed through two distinct pathways: a “what” pathway that extracts and learns the local visual signatures (landmarks) of a visual scene and a “where” pathway that computes their azimuth. These two pieces of information are then merged to build a visuospatial code that is characteristic of the place where the visual scene was perceived. Three main contributions are presented in this article: 1) the LPMP model is studied and compared with NetVLAD and CoHog, two state-of-the-art VPR models; 2) a test benchmark for the evaluation of VPR models according to the type of environment traveled is proposed based on the Oxford car dataset; and 3) the impact of the use of a novel detector leading to an uneven paving of an environment is evaluated in terms of the localization performance and compared to a regular paving. Our experiments show that the LPMP model can achieve comparable or better localization performance than NetVLAD and CoHog.

New particle formation event detection with Mask R-CNN

Abstract. Atmospheric new particle formation (NPF) is an important source of climate-relevant aerosol particles which has been observed at many locations globally. To study this phenomenon, the first step is to identify whether an NPF event occurs or not on a given day. In practice, NPF event identification is performed visually by classifying the NPF event or non-event days from the particle number size distribution surface plots. Unfortunately, this day-by-day visual classification is time-consuming and labor-intensive, and the identification process renders subjective results. To detect NPF events automatically, we regard the visual signature (banana shape) which has been observed all over the world in NPF surface plots as a special kind of object, and a deep learning model called Mask R-CNN is applied to localize the spatial layouts of NPF events in their surface plots. Utilizing only 358 human-annotated masks on data from the Station for Measuring Ecosystem–Atmosphere Relations (SMEAR) II station (Hyytiälä, Finland), the Mask R-CNN model was successfully generalized for three SMEAR stations in Finland and the San Pietro Capofiume (SPC) station in Italy. In addition to the detection of NPF events (especially the strongest events), the presented method can determine the growth rates, start times, and end times for NPF events automatically. The automatically determined growth rates agree with the manually determined growth rates. The statistical results validate the potential of applying the proposed method to different sites, which will improve the automatic level for NPF event detection and analysis. Furthermore, the proposed automatic NPF event analysis method can minimize subjectivity compared with human-made analysis, especially when long-term data series are analyzed and statistical comparisons between different sites are needed for event characteristics such as the start and end times, thereby saving time and effort for scientists studying NPF events.

Encoding of Luminescent Ink Markers Using Low-Level Data Fusion and Chemometrics

The identification and analysis of documentary fraud is always a challenge for forensic science. Document analysis has proven to be an important branch of forensics in elucidating the authenticity of documents. The development and incorporation of luminescent inks in authentic documents have proved to be an excellent security feature. This paper purposes the use of a possible luminescent ink marker for anti-counterfeiting applications, aiming to create a document encoding process that is simple, robust, sensitive, and non-destructive. Since luminescent inks markers provide a visual, chemical, and spectral signature, and can be easily detected by using a UV lamp, the aid of unsupervised chemometric tools makes it possible to differentiate the luminescent markers inserted in the ink. Unsupervised models of principal component analysis (PCA) and K-mean were successful in correctly associating marked inks with their respective pure markers, while a supervised classification model based on partial least squares discriminant analysis (PLS-DA) correctly classified all samples from the prediction set and the blind test samples. For comparison, a soft independent modeling of class analogy (SIMCA) model was also built, which despite showing a misclassified sample it is also a strong candidate for future applications.

Preliminary study on the aesthetic preference for taillight shape design

The shape of a taillight not only improves the safety features of cars, but also provides a vast range of design options for vehicle designers. Moreover, it is regarded as a powerful design element of cars. This study aims to investigate the aesthetic preferences of consumers for taillight shape design and provide insights on visually appealing shapes for vehicle designers and manufacturers. The experiment used in this study consisted of bottom-up and top-down attentional processes that used an integrated eye-tracking method with self-report. The results showed that a taillight shape with contour features or medium length was more aesthetically preferred by consumers. Additionally, horizontal taillight shapes were more preferred compared to vertical taillight shapes. Taillight shape design insights can help in creating visually appealing rear-end designs by not only enhancing the aesthetic appearance of vehicles but also by helping in creating a unique visual signature for the brand.

Active learning via adaptive weighted uncertainty sampling applied to additive manufacturing

The acquisition of large amounts of high-dimensional data is becoming prevalent in additive manufacturing, arising from sensor integration in such processes. Although supervised machine learning has gained popularity to predict process quality from such data, annotation can be time-consuming and labor intensive. Active learning is a sub-field within machine learning concerned with maximizing model performance using the least amount of annotated data. The focus of this study is twofold. Firstly, a novel active learning method is introduced, called adaptive weighted uncertainty sampling (AWUS) which balances uncertainty with random sampling using the model change between active learning iterations. This way, AWUS is able to adapt from exploration of the instance space to exploitation of model knowledge throughout the active learning process. Secondly, a novel feature extraction and classification method is proposed for directed-energy-deposition additive manufacturing processes which predicts image and process quality based upon the visibility of the melt-pool using the visual signature of occluding features in the image. AWUS is compared against 6 state-of-the-art query strategies on 28 datasets from the OpenML database and 8 in-situ machine vision recordings of directed-energy-deposition processes using 4 different classifiers. The results show that AWUS outperforms the state-of-the-art across the board of the datasets, classifiers and active learning batch sizes. Furthermore, the application of AWUS reduces the amount of necessary annotations by 20–70% in 90% of our experiments compared to random sampling.

Investigating Malware Propagation and Behaviour Using System and Network Pixel-Based Visualisation

Malicious software, known as malware, is a perpetual game of cat and mouse between malicious software developers and security professionals. Recent years have seen many high profile cyber attacks, including the WannaCry and NotPetya ransomware attacks that resulted in major financial damages to many businesses and institutions. Understanding the characteristics of such malware, including how malware can propagate and interact between systems and networks is key for mitigating these threats and containing the infection to avoid further damage. In this study, we present visualisation techniques for understanding the propagation characteristics in dynamic malware analysis. We propose the use of pixel-based visualisations to convey large-scale complex information about network hosts in a scalable and informative manner. We demonstrate our approach using a virtualised network environment, whereby we can deploy malware variants and observe their propagation behaviours. As a novel form of visualising system and network activity data across a complex environment, we can begin to understand visual signatures that can help analysts identify key characteristics of the malicious behaviours, and, therefore, provoke response and mitigation against such attacks.

The Nature of Black Hole Shadows

Abstract A distinct visual signature occurs in black holes that are surrounded by optically thin and geometrically thick emission regions. This signature is a sharp-edged dip in brightness that is coincident with the black hole’s shadow, which is the projection of the black hole’s unstable-photon region on the observer’s sky. We highlight two key mechanisms that are responsible for producing the sharp-edged dip: (i) the reduction of intensity observed in rays that intersect the unstable-photon region, and thus the perfectly absorbing event horizon, versus rays that do not (blocking); and (ii) the increase of intensity observed in rays that travel along extended, horizon-circling paths near the boundary of the unstable-photon region (path-lengthening). We demonstrate that the black hole shadow is a distinct phenomenon from the photon ring, and that models exist in which the former may be observed but not the latter. Additionally, we show that the black hole shadow and its associated visual signature differ from the more model-dependent brightness depressions associated with thin-disk models because the blocking and path-lengthening effects are quite general for geometrically thick and optically thin emission regions. Consequentially, the black hole shadow is a robust and fairly model-independent observable for accreting black holes that are in the deep sub-Eddington regime, such as low-luminosity active galactic nuclei.

The visual signature of non-understanding: A systematic replication of McDonough, Trofimovich, Lu, and Abashidze (2019)

AbstractThis replication study seeks to extend the generalizability of an exploratory study (McDonough et al., 2019) that identified holds (i.e., temporary cessation of dynamic movement by the listener) as a reliable visual cue of non-understanding. Conversations between second language (L2) English speakers in the Corpus of English as a Lingua Franca Interaction (CELFI; McDonough &amp; Trofimovich, 2019) with non-understanding episodes (e.g., pardon?, what?, sorry?) were sampled and compared with understanding episodes (i.e., follow-up questions). External raters (N = 90) assessed the listener's comprehension under three rating conditions: +face/+voice, −face/+voice, and +face/−voice. The association between non-understanding and holds in McDonough et al. (2019) was confirmed. Although raters distinguished reliably between understanding and non-understanding episodes, they were not sensitive to facial expressions when judging listener comprehension. The initial and replication findings suggest that holds remain a promising visual signature of non-understanding that can be explored in future theoretically- and pedagogically-oriented contexts.

Robust license plate signatures matching based on multi-task learning approach

Identifying a car by its License Plate (LP) is a critical task in many applications, such as travel time estimation, vehicle re-identification, automatic toll collection, etc. Therefore, matching them must be as accurate as possible. This research proposes a novel deep neural network based method and its learning strategy for LP matching (LPM), which is originally from the cognitive-psychology-inspired unified objective function. The proposed method uses a deep Convolutional Neural Network (CNN) model to extract effective visual signature of the LP image. It exploits the multi-task learning approach to optimize the model, which combines two different tasks: (a) parallel letters recognition to transcribe the image-text contents and (b) image classification to classify the distinct LPs. Moreover, it takes profit from the use of image augmentation techniques. The proposed method is evaluated on three datasets of different characteristics. One of them was collected in this research and will be released publicly. The obtained results show that the proposed method performs better than the state-of-the art methods based on the commonly used evaluation metrics and computation time.

America’s Name Baptized on a Globe in 1510. Leonardo da Vinci’s Blueprint for the Jagiellonian Armillary Sphere Discovered.

The treasure room at the Collegium Maius of the Jagiellonian Museum in Cracow contains an armillary sphere dating from 1510. This scientific object of excellent French workmanship contains the Jagiellonian Globe, which looks surprisingly like the Lenox Globe, a sibling of the da Vinci Globe. The fact that the Jagiellonian Globe is mounted in an armillary sphere supports the hypothesis that the Lenox, cast from reddish copper, was most likely the central part of a lost armillary sphere. A horologist copied the cartography of the Lenox Globe. It is hypothesized that he also copied the decorative artistic design of the armillary sphere as a blueprint. Using primary data in his research, the author describes the compelling Jagiellonian instrument. He concentrates on aspects of epigraphy, toponymy, orthography, iconography, cosmography, ornamental history, visual arts, heraldry, kinematics, geometry, didactics and astronomy. The methodology used is based on analogy in the arts, stemmatics, cartographic, historiographical and comparative analysis based on the latest 3D photographic scanning technology of the Lazarus Project of the University of Rochester. Furthermore, more than 40 international experts and researchers contained in the list of acknowledgements assisted in making this research possible. The author attributes the Jagiellonian Globe to Jean Coudray, Early Modern horologist active for successive French Kings in Blois. This is substantiated by a monogram. It is a capital letter C next to a reversed half-Moon on the bottom of the Jagiellonian Globe. The author provides key evidence that this French horologist constructed the instrument between 1507 and 1510 based on a model armillary sphere. In making the Jagiellonian, Jean Coudray added the latest cartographic news in the form of a Latin phrase “America noviter reperta” thereby baptizing the name of America for the first time in Early Modern history on a three-dimensional object. Compelling arguments and chronological evidence are offered by the orthography, nomenclature, applied old French dimensions, iconography of the instrument and the unique cartography of the terrestrial globe contained at its center. The specific didactic scheme of this universal armillary sphere, in addition to the whirlpools adjacent to the orb, bear the visual signature of Leonardo da Vinci. However, the French horologist was not only influenced by stylistic Renaissance and didactical aspects used by Leonardo. He copied them. These include the mirroring of the Roman numerals and of the order of the hour band on the Equator reflecting the concaveness of the object based on a Vitruvian design. In addition, the unique design of the throne of the armillary sphere is influenced by Leonardo’s kinetics. The two scrolls of the throne contained in a newly developed design, in this case one of life-giving whirlpools, echoes the kinetic untamed energy and power of nature and of the oceans in particular. The iconography of the world sphere, the cosmic egg, in between the two scrolls means perfection, the primordial form which contains all the possibilities of all the forms as Plato’s animus mundi, the soul of the world. The anonymous large mountainous island mirroring the Regio Pathalis of Pliny and Bacon is used in the cartography of the da Vinci, Lenox and Jagiellonian Globe. Finally, the author offers evidence of a bibliographical reference to a specific astronomical clock instrument just like the Jagiellonian Armillary Sphere. This reference is listed in a contemporary French inventory of the famous Florimond Robertet, notorious client of Leonardo da Vinci, dating from 1532.

Engaging Patients in Integrated Behavioral Health and Primary Care: A Technology-Based System to Facilitate Behavioral Health Screening for Patients in Rural and Underserved Areas.

Integrating behavioral health into primary care can increase patient access to treatments for behavioral health disorders, especially in rural areas. The first step of integrated care implementations is the need to identify at-risk patients and effectively communicate patient screening results to the care team. The use of technology can facilitate patient screening and incorporate screening data into electronic health records. This paper describes the development of a tablet-based screening system to facilitate behavioral health integration in a rural primary care clinic, discusses the preliminary impact of digital screening on workflows, and provides recommendations for the use of technology for screening. A tablet-based assessment was developed to screen patients for behavioral health concerns and was implemented within a rural primary care clinic. The system was created using the Visual Signature Capture system, which integrates directly with the electronic health record. Following the initial assessment, patients are screened as frequently as every 30 days with a mini-screen assessment, consisting of multiple screening tools for mental health and substance use. The tablet-based screening system had a positive impact on clinician workflows and the overall effectiveness of clinic operations. This system supports providers in addressing the behavioral health needs of patients that often go unrecognized in primary care.