Unified Image Research Articles

Label-free biochemical quantitative phase imaging with mid-infrared photothermal effect

Label-free optical imaging is valuable in biology and medicine because of its non-destructive nature. Quantitative phase imaging (QPI) and molecular vibrational imaging (MVI) are the two most successful label-free methods, providing morphological and biochemical information, respectively. These techniques have enabled numerous applications as they have matured over the past few decades; however, their label-free contrasts are inherently complementary and difficult to integrate due to their reliance on different light–matter interactions. Here we present a unified imaging scheme with simultaneous and in situ acquisition of quantitative phase and molecular vibrational contrasts of single cells in the QPI framework using the mid-infrared photothermal effect. The robust integration of subcellular morphological and biochemical label-free measurements may enable new analyses, especially for studying complex and fragile biological phenomena such as drug delivery, cellular disease, and stem cell development, where long-time observation of unperturbed cells is needed under low phototoxicity.

Beyond Vision: A Multimodal Recurrent Attention Convolutional Neural Network for Unified Image Aesthetic Prediction Tasks

Over the past few years, image aesthetic prediction has attracted increasing attention because of its wide applications, such as image retrieval, photo album management and aesthetic-driven image enhancement. However, previous studies in this area only achieve limited success because 1) they primarily depend on visual features and ignore textual information. 2) they tend to focus equally on to each part of images and ignore the selective attention mechanism. This paper overcomes these limitations by proposing a novel multimodal recurrent attention convolutional neural network (MRACNN). More specifically, the MRACNN consists of two streams: the vision stream and the language stream. The former employs the recurrent attention network to tune out irrelevant information and focuses on some key regions to extract visual features. The latter utilizes the Text-CNN to capture the high-level semantics of user comments. Finally, a multimodal factorized bilinear (MFB) pooling approach is used to achieve effective fusion of textual and visual features. Extensive experiments demonstrate that the proposed MRACNN significantly outperforms state-of-the-art methods for unified aesthetic prediction tasks: (i) aesthetic quality classification; (ii) aesthetic score regression; and (iii) aesthetic score distribution prediction.

Foveation Pipeline for 360° Video-Based Telemedicine.

Pan-tilt-zoom (PTZ) and omnidirectional cameras serve as a video-mediated communication interface for telemedicine. Most cases use either PTZ or omnidirectional cameras exclusively; even when used together, images from the two are shown separately on 2D displays. Conventional foveated imaging techniques may offer a solution for exploiting the benefits of both cameras, i.e., the high resolution of the PTZ camera and the wide field-of-view of the omnidirectional camera, but displaying the unified image on a 2D display would reduce the benefit of “omni-” directionality. In this paper, we introduce a foveated imaging pipeline designed to support virtual reality head-mounted displays (HMDs). The pipeline consists of two parallel processes: one for estimating parameters for the integration of the two images and another for rendering images in real time. A control mechanism for placing the foveal region (i.e., high-resolution area) in the scene and zooming is also proposed. Our evaluations showed that the proposed pipeline achieved, on average, 17 frames per second when rendering the foveated view on an HMD, and showed angular resolution improvement on the foveal region compared with the omnidirectional camera view. However, the improvement was less significant when the zoom level was 8× and more. We discuss possible improvement points and future research directions.

Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity

In this paper, we propose a fast unified image fusion network based on proportional maintenance of gradient and intensity (PMGI), which can end-to-end realize a variety of image fusion tasks, including infrared and visible image fusion, multi-exposure image fusion, medical image fusion, multi-focus image fusion and pan-sharpening. We unify the image fusion problem into the texture and intensity proportional maintenance problem of the source images. On the one hand, the network is divided into gradient path and intensity path for information extraction. We perform feature reuse in the same path to avoid loss of information due to convolution. At the same time, we introduce the pathwise transfer block to exchange information between different paths, which can not only pre-fuse the gradient information and intensity information, but also enhance the information to be processed later. On the other hand, we define a uniform form of loss function based on these two kinds of information, which can adapt to different fusion tasks. Experiments on publicly available datasets demonstrate the superiority of our PMGI over the state-of-the-art in terms of both visual effect and quantitative metric in a variety of fusion tasks. In addition, our method is faster compared with the state-of-the-art.

The unified imaging condition of Fourier-transform ghost imaging in the far field

In general, the unequal distance between the two detectors and the thermal light source can lead to poor imaging quality in the Fourier-transform ghost diffraction (FGD) system, while the rule does not hold in the far field. Based on the numerical simulations and experimental results, the imaging condition of the FGD pattern is analyzed in the far field. It is shown that the condition is quite different from that in the near field. When the difference between the distance from the light source to the reference detector and the one from the object to the light source are in the far field, the better diffraction pattern can also be kept when the equivalent distance from the object to the test detector is in the far field, although the lengths of the test path and the reference path are not equal.

Boundaries Extend and Contract in Scene Memory Depending on Image Properties.

Boundary extension, a memory distortion in which observers consistently recall a scene with visual information beyond its boundaries, is widely accepted across the psychological sciences as a phenomenon revealing fundamental insight into memory representations [1-3], robust across paradigms [1, 4] and age groups [5-7]. This phenomenon has been taken to suggest that the mental representation of a scene consists of an intermingling of sensory information and a schema that extrapolates the views of a presented scene [8], and it has been used to provide evidence for the role of the neocortex [9] and hippocampus [10, 11] in the schematization of scenes during memory. However, the study of boundary extension has typically focused on object-oriented images that are not representative of our visuospatial world. Here, using a broad set of 1,000 images tested on 2,000 participants in a rapid recognition task, we discover "boundary contraction" as an equally robust phenomenon. Further, image composition largely drives whether extension or contraction is observed-although object-oriented images cause more boundary extension, scene-oriented images cause more boundary contraction. Finally, these effects also occur during drawing tasks, including a task with minimal memory load-when participants copy an image during viewing. Collectively, these results show that boundary extension is not a universal phenomenon and put into question the assumption that scene memory automatically combines visual information with additional context derived from internal schema. Instead, our memory for a scene may be largely driven by its visual composition, with a tendency to extend or contract the boundaries equally likely.

A Postmodern Thanatic Triad: Crisis, Pornography and Renaissance of Death

In every age we can find a different attitude of man towards the end of life, instanced in varied stances and imaginations regarding death. In postmodernism, which makes a broader context for this work, it is impossible to present one unified image of death, because the attitude of contemporary men towards death and the visual forms of it are greatly varied. The subject of my analysis is a presentation of death shown in three occurrences the crisis, the pornography and the renaissance of death which combine to form the postmodern thanatic triad.

From Sheep to Chic: Reframing the Australian Wool Story

ABSTRACTBetween 1946 and 1974, the Australian Wool Board (AWB) increased its wool promotion in an attempt to maintain its leading role in Australia's economic development. Promotional activities of the AWB, and the screening of commercially produced newsreels and documentaries in cinemas, used fashion to foster a new value and understanding of wool. These marketing activities, mainly independent from each other, presented a unified image of wool and fashion. This article proposes that the recasting of Australian wool's cultural heritage in images of fashion parades of wool garments contributed to the continuity of its cultural tradition while integrating fashion as a modern value within national culture and identity. The article examines 14 newsreels screened in Australian cinemas between 1946 and 1974 to establish the connection between Australian national identity and wool via the reinvention of the wool narrative.

«АРХІВ ІСТОРІЇ УНІЇ» ЯК ІНТЕГРАЛЬНА ЧАСТИНА ВІЗАНТІЙСЬКОЇ БІБЛІОТЕКИ Й АРХІВУ КНИГОЗБІРНІ «СТУДІОН»

On the basis of a large array of archival documents and historiographical base, the activity of Klymentii Sheptytskyi as abbot of the Holy Assumption Univ Lavra is highlighted. The analysis of this suggests that the abbot Klymentii not only contributed to the restoration of the model of the convents of the Studio Charter, but thanks to the author’s messages and works became the charismatic law-maker of this model. The author found out that the monasteries of the studio, headed by the abbot Klymentii, were able to: consolidate the Ukrainian emigrants, resist the denationalization from both the Russian and Polish sides; to spread education among young people and children, which has influenced the process of education in the religious-patriotic spirit of the Eastern Galician society; to create religious and ecclesiastical periodicals and a network of monastic libraries, which played a significant role in the development of Christian and national ideas among the local population; revive sacred art, which greatly enriched the national culture; support and care for the vulnerable, including orphans, the sick and the poor. In the article it is proved that the activity of abbot Klymentii (Sheptytskyi), first, became an important factor and criterion of moral and spiritual enrichment of Ukrainians, secondly, is a clear statement that the GCC stood on the principles of Christian morality, national and cultural tolerance and upholding — religious interests of Ukrainian, and the monasteries of the Studio Charter became one of the greatest expressors, the guarantor of the realization of national and cultural interests of Ukrainians in Eastern Galicia.The paper deals with organizational principles and practical work of the «Union history archive» (UHA), its functions, ways and dynamics of additions, structure and thematic policy of archival and library funds, personnel issue. The role of UHA founder Galician Metropolitan Andrey Sheptytskyi (1865‒1944) remains an uninvestigated issue in the context of interwar archival institution studies. The research based on newly discovered primary sources and materials of Vasyl Stefanyk National Scientific Library of Ukraine in Lviv, Central State Historical Archive of Ukraine in Lviv, and State Archive of Lviv oblast. Having a set of newly discovered archival documents: 381 «books of introductions», reports on expeditions, financial receipts, etc., as well as the correspondence of Metropolitan Andrey Sheptytskyi, director and staff of UHA, a unified image of UHA in its historical development is reproduced. In particular, the main aspects of the current work of the UHA, the role of the Metropolitan and director Ivan Shendryk in coordinating the processes of acquisition, monetary evaluation, purchasing of rarities with the participation of freight forwarders O. Tsynkalovsky, B. Olkhivsky and other individuals in searching for historical (church) written and printed monuments, as well as art and archaeological exhibits; establishing contacts with residents of Volyn, Kholm, Polissya, Podlasie, as well as Lithuania, Belarus, Russia and others. One of the well-established forms of UHA acquisition has been studied — the purchasing of cultural monuments in bookstores, antique shops, «on the market» and auctions that allows tracing the dynamics of UHA acquisition. The organizational activities of Metropolitan A. Sheptytskyi and the archival research works financially provided by him in the archives of the Vatican, the Peremyshl Chapter, the Archive of Ancient Acts in Warsaw, the Ossolinski National Institution in Lviv and others institutions have been studied. The structure and thematic palette of UHA components are revealed: the archive of ancient acts (before the 1917 revolution) and the archive of new acts (from 1917), book collections, including the «archival library». The initiatives of the founder of the Union History Archive, Metropolitan Andrey Sheptytskyi, on the reorganization and inclusion of UHA funds in the archive-library complex of the «Studion» book collection as an integral part of the Byzantine library and archive are highlighted. The activity of Metropolitan Andrey Sheptytskyi in preventing the destruction of the whole layer of culture and enrichment of the national treasury of Ukraine is presented on the example of coverage of the work of UHA. Keywords: Archives, Library, Metropolitan Andrey Sheptytskyi, Book collection, Union, Historical monuments, Manuscripts.

Single-molecule methods in structural DNA nanotechnology.

Single molecules can now be visualised with unprecedented precision. As the resolution of single-molecule experiments improves, so too does the breadth, quantity and quality of information that can be extracted using these methodologies. In the field of DNA nanotechnology, we use programmable interactions between nucleic acids to generate complex, multidimensional structures. We can use single-molecule techniques - ranging from electron and fluorescence microscopies to electrical and force spectroscopies - to report on the structure, morphology, robustness, sample heterogeneity and other properties of these DNA nanoconstructs. In this Tutorial Review, we will detail how complementarity between static and dynamic single-molecule techniques can provide a unified image of DNA nanoarchitectures. The single-molecule methods that we discuss provide unprecedented insight into chemical and structural behaviour, yielding not just an average outcome but reporting on the distribution of values, ultimately showing how bulk properties arise from the collective behaviour of individual structures. As the fields of both DNA nanotechnology and single-molecule characterisation intertwine, a feedback loop is generated between disciplines, providing new opportunities for the development and operation of DNA-based materials as sensors, delivery vehicles, machinery and structural scaffolds.

The image of Crimea among British travelers Edward Clarke and Reginald Heber at the turn of the 18th-19th centuries

This paper analyses two books by British travellers offering accounts of Crimea in the first decades of the Russian period in its history. Crimea became a stage in Western Grand Tour, offering a possibility to view and discuss different phenomena: Mediterranean environment, cultural heritage sites, multiethnic populations confessing different religions, the change of Crimea’s political status, and the first results of Russia’s attempts of its integration. The comparison of these two travelogues with other sources and the materials supplied by current researches has uncovered who the British mind interpreted Crimean realities. The travellers created unified image of Crimea featuring its past, present, and future. The travelogues under analysis uncover the features of researchers’ thinking in the period of transition from the Enlightenment to the Romanticism. This way, the notion of ethnic processes actually restricted to the search for modern parallels of ancient ethnic names. The books under study reflect a complicated and controversial process of Crimea’s integration into the Russian Empire. Heber considered the future as economic progress and therefore thought it necessary to develop Crimean trade, infrastructure, and economy, building them into all-Russia and all-Europe network. Clarke’s opinion of Russia was distinctly negative, therefore he thought desirable to ‘return’ Crimea to the Ottomans. The travellers created several stereotypes, such as the ideas of ‘earthly paradise’ in Crimea, ‘Tatar laziness,’ ‘golden age’ of the Crimean khanate, or ‘barbarous destruction’ of cultural heritage monuments by Russians, still existing in Western mind.

An Object-Based Strategy for Improving the Accuracy of Spatiotemporal Satellite Imagery Fusion for Vegetation-Mapping Applications

Spatiotemporal data fusion is a key technique for generating unified time-series images from various satellite platforms to support the mapping and monitoring of vegetation. However, the high similarity in the reflectance spectrum of different vegetation types brings an enormous challenge in the similar pixel selection procedure of spatiotemporal data fusion, which may lead to considerable uncertainties in the fusion. Here, we propose an object-based spatiotemporal data-fusion framework to replace the original similar pixel selection procedure with an object-restricted method to address this issue. The proposed framework can be applied to any spatiotemporal data-fusion algorithm based on similar pixels. In this study, we modified the spatial and temporal adaptive reflectance fusion model (STARFM), the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) and the flexible spatiotemporal data-fusion model (FSDAF) using the proposed framework, and evaluated their performances in fusing Sentinel 2 and Landsat 8 images, Landsat 8 and Moderate-resolution Imaging Spectroradiometer (MODIS) images, and Sentinel 2 and MODIS images in a study site covered by grasslands, croplands, coniferous forests, and broadleaf forests. The results show that the proposed object-based framework can improve all three data-fusion algorithms significantly by delineating vegetation boundaries more clearly, and the improvements on FSDAF is the greatest among all three algorithms, which has an average decrease of 2.8% in relative root-mean-square error (rRMSE) in all sensor combinations. Moreover, the improvement on fusing Sentinel 2 and Landsat 8 images is more significant (an average decrease of 2.5% in rRMSE). By using the fused images generated from the proposed object-based framework, we can improve the vegetation mapping result by significantly reducing the “pepper-salt” effect. We believe that the proposed object-based framework has great potential to be used in generating time-series high-resolution remote-sensing data for vegetation mapping applications.

The tripartite architecture of the eukaryotic integral membrane protein zinc metalloprotease Ste24.

Ste24 enzymes, a family of eukaryotic integral membrane proteins, are zinc metalloproteases (ZMPs) originally characterized as “CAAX proteases” targeting prenylated substrates, including a‐factor mating pheromone in yeast and prelamin A in humans. Recently, Ste24 was shown to also cleave nonprenylated substrates. Reduced activity of the human ortholog, HsSte24, is linked to multiple disease states (laminopathies), including progerias and lipid disorders. Ste24 possesses a unique “α‐barrel” structure consisting of seven transmembrane (TM) α‐helices encircling a large intramembranous cavity (~14 000 Å3). The catalytic zinc, coordinated via a HExxH…E/H motif characteristic of gluzincin ZMPs, is positioned at one of the cavity's bases. The interrelationship between Ste24 as a gluzincin, a long‐studied class of soluble ZMPs, and as a novel cavity‐containing integral membrane protein protease has been minimally explored to date. Informed by homology to well‐characterized soluble, gluzincin ZMPs, we develop a model of Ste24 that provides a conceptual framework for this enzyme family, suitable for development and interpretation of structure/function studies. The model consists of an interfacial, zinc‐containing “ZMP Core” module surrounded by a “ZMP Accessory” module, both capped by a TM helical “α‐barrel” module of as yet unknown function. Multiple sequence alignment of 58 Ste24 orthologs revealed 38 absolutely conserved residues, apportioned unequally among the ZMP Core (18), ZMP Accessory (13), and α‐barrel (7) modules. This Tripartite Architecture representation of Ste24 provides a unified image of this enzyme family.

A Gated Peripheral-Foveal Convolutional Neural Network for Unified Image Aesthetic Prediction

Learning fine-grained details is a key issue in image aesthetic assessment. Most of the previous methods extract the fine-grained details via random cropping strategy, which may undermine the integrity of semantic information. Extensive studies show that humans perceive fine-grained details with a mixture of foveal vision and peripheral vision. Fovea has the highest possible visual acuity and is responsible for seeing the details. The peripheral vision is used for perceiving the broad spatial scene and selecting the attended regions for the fovea. Inspired by these observations, we propose a Gated Peripheral-Foveal Convolutional Neural Network (GPF-CNN). It is a dedicated double-subnet neural network, i.e. a peripheral subnet and a foveal subnet. The former aims to mimic the functions of peripheral vision to encode the holistic information and provide the attended regions. The latter aims to extract fine-grained features on these key regions. Considering that the peripheral vision and foveal vision play different roles in processing different visual stimuli, we further employ a gated information fusion (GIF) network to weight their contributions. The weights are determined through the fully connected layers followed by a sigmoid function. We conduct comprehensive experiments on the standard AVA and Photo.net datasets for unified aesthetic prediction tasks: (i) aesthetic quality classification; (ii) aesthetic score regression; and (iii) aesthetic score distribution prediction. The experimental results demonstrate the effectiveness of the proposed method.

An Efficient Compressive Convolutional Network for Unified Object Detection and Image Compression

This paper addresses the challenge of designing efficient framework for real-time object detection and image compression. The proposed Compressive Convolutional Network (CCN) is basically a compressive-sensing-enabled convolutional neural network. Instead of designing different components for compressive sensing and object detection, the CCN optimizes and reuses the convolution operation for recoverable data embedding and image compression. Technically, the incoherence condition, which is the sufficient condition for recoverable data embedding, is incorporated in the first convolutional layer of the CCN model as regularization; Therefore, the CCN convolution kernels learned by training over the VOC and COCO image set can be used for data embedding and image compression. By reusing the convolution operation, no extra computational overhead is required for image compression. As a result, the CCN is 3.1 to 5.0 fold more efficient than the conventional approaches. In our experiments, the CCN achieved 78.1 mAP for object detection and 3.0 dB to 5.2 dB higher PSNR for image compression than the examined compressive sensing approaches.

Unified radio interferometric calibration and imaging with joint uncertainty quantification

The data reduction procedure for radio interferometers can be viewed as a combined calibration and imaging problem. We present an algorithm that unifies cross-calibration, self-calibration, and imaging. Because it is a Bayesian method, this algorithm not only calculates an estimate of the sky brightness distribution, but also provides an estimate of the joint uncertainty which entails both the uncertainty of the calibration and that of the actual observation. The algorithm is formulated in the language of information field theory and uses Metric Gaussian Variational Inference (MGVI) as the underlying statistical method. So far only direction-independent antenna-based calibration is considered. This restriction may be released in future work. An implementation of the algorithm is contributed as well.

Comparative Efficiency of Methods of Cooperative Processing of Imagery Information

A problem of detection of 3D objects on the basis of imagery information provided by two or three different sensors is considered. The efficiency of object detection with the use of algorithms of integration and fusion of information from different sensors and also by means of formation of a unified synthetic image from two initial images is estimated. Conditions are determined at which the use of the data integration algorithm and the method of forming a single synthetic image provides a lower efficiency as compared to the use of only one image among those available for detection.

Unified wave field retrieval and imaging method for inhomogeneous non-reciprocal media.

Acoustic imaging methods often ignore multiple scattering. This leads to false images in cases where multiple scattering is strong. Marchenko imaging has recently been introduced as a data-driven way to deal with internal multiple scattering. Given the increasing interest in non-reciprocal materials, both for acoustic and electromagnetic applications, a modification to the Marchenko method is proposed for imaging such materials. A unified wave equation is formulated for non-reciprocal materials, exploiting the similarity between acoustic and electromagnetic wave phenomena. This unified wave equation forms the basis for deriving reciprocity theorems that interrelate wave fields in a non-reciprocal medium and its complementary version. Next, these theorems are reformulated for downgoing and upgoing wave fields. From these decomposed reciprocity theorems, representations of the Green's function inside the non-reciprocal medium are derived in terms of the reflection response at the surface and focusing functions inside the medium and its complementary version. These representations form the basis for deriving a modified version of the Marchenko method to retrieve the wave field inside a non-reciprocal medium and to form an image, free from artefacts related to multiple scattering. The proposed method is illustrated at the hand of the numerically modeled reflection response of a horizontally layered medium.

Self as an Aesthetic Effect.

Mainstream psychology has assumed a notion of the self that seems to rest on a substantialist notion of the psyche that became predominant despite important critical theories about the self. Although cultural psychology has recognized the diverse, dialogical, historical, narrative, and performative nature of self, as opposed to the idea of self as entity, it is not clear how it accounts for the phenomenological experience of self as a unified image. In this paper, we offer a theoretical contribution to developing the implications of a genetic approach to self in cultural psychology, taking into account an otherwise overlooked dimension: art and aesthetics. We draw on the work of classical authors relevant to cultural psychology, who, although geopolitically and theoretically diverse, are concerted in understanding human psychological life as part of a living process of becoming: James, Mead, Dewey, Vygotsky, Bakhtin, and Vološinov. Overall, the hypothesis developed throughout the paper is that self is produced within psychological individuation as an effect of the aesthetic activity involved in everyday discursive life. We deepen the ideas that self is not an entity but a process of open becoming and that cultural life entails a radical experience of alterity, but we recognize the psychological importance of the sense of unity and closure generated in this process. We argue that self entails not only the process of becoming but also an aesthetical effect of unity in becoming. Self as an aesthetic effect emphasizes the self as a discursive and technical process of production, involving a product that, despite not being a finished entity, is felt as unitary and as mine by virtue of a specific transformation of experience. We thus propose to define self, on one level, as an epistemological category that points to the paradoxes of identity and agency in psychological individuation, and, on a different level, as a twofold operation that makes possible the subjective experience of a constitutive effort as much as a transient but experienced identity or agency.

Learning to Find Unpaired Cross-Spectral Correspondences.

We present a deep architecture and learning framework for establishing correspondences across cross-spectral visible and infrared images in an unpaired setting. To overcome the unpaired cross-spectral data problem, we design the unified image translation and feature extraction modules to be learned in a joint and boosting manner. Concretely, the image translation module is learned only with the unpaired cross-spectral data, and the feature extraction module is learned with an input image and its translated image. By learning two modules simultaneously, the image translation module generates the translated image that preserves not only the domain-specific attributes with separate latent spaces but also the domain-agnostic contents with feature consistency constraint. In an inference phase, the cross-spectral feature similarity is augmented by intra-spectral similarities between the features extracted from the translated images. Experimental results show that this model outperforms the state-of-the-art unpaired image translation methods and cross-spectral feature descriptors on various visible and infrared benchmarks.