Image Decoding Research Articles

Characteristics of air temperature on the southern and southeastern slopes of the Greater Caucasus in the modern period

In the paper, the characteristics of air temperature in the south part of the Greater Caucasus region and the effects of regional climate changes on this area in the modern period were studied. In the research, the air temperature observation data of 8 hydrometeorological stations operating in the region for the years 1961-2022 were used, and the modern classification of the average monthly, seasonal, annual and altitude time-space distributions of temperature was given.Based on the data of hydrometeorological stations in the south and southeast piedmonts of the Greater Caucasus Mountains, an electronic map was prepared for the distribution of average annual air temperature indicators on the territory, and for determining the indicators of areas without data. DEM (Digital Elevation Model) files obtained fromthe decoding of satellite images (satellite data) were used in the prepared map. The research shows that July and August are the hottest months when the temperature is 22.80C, while January is the coldest by temperature about 0.50C in this region. In this part of the Greater Caucasus province, in the cold months of the year, the average monthly temperature is indicating below 00C in the area located above the moderately mountainous part of the region. The dynamics of the average indicators of temperature of the multi-year period (1961-2022) are expressed by a positive trend. The average annual temperature distribution map of the territory shows that the air temperature is 15.00C in the low mountainous part of the region, while it decreases upwards and, according to the adiabatic law, equals 00C at altitudes above 3500 m. According to the climate studies conducted in the region, compared to the years 1981-2010, it was found that there was a temperature increase of 1.20C in the mountainous region in 2011-2022. Also, a comparative analysis of 10-year averages of air temperature trends, using the Holt- Winters model, shows that temperatures in 2100 are expected to increase by 3.00C compared to 2030. In the region, climate changes that worsen every year lead to water scarcity, desertification, drought, etc. and can expand the area of recurrence of environmental crises and destructive natural events. This research can be used in future climate change mitigation , climate atlases, infrastructure, agricultural and industrial projects in the region.

Open-Vocabulary Part-Level Detection and Segmentation for Human–Robot Interaction

Object detection and segmentation have made great progress in robotic application missions. However, intelligent agents require fine-grained recognition algorithms rather than object-level and language instructions to enhance the interaction between humans and robots. To improve the robot’s interactivity in the process of the robot response to language instructions, we propose a method for part-level detection and segmentation by exploiting vision language models. In this approach, Swin Transformer is introduced in the image encoder for extracting image features, and FPNs (Feature Pyramid Networks) are modified to better process the features from Swin Transformer. Next, the image decoder is proposed for model aligning between the image features and text embeddings for achieving human–robot interaction via language. Finally, we verify that the text embeddings are impacted by the command of input and that different prompt templates also affect classification. The method proposed in this paper, which is validated on two datasets (PartImagePart and Pascal Part), possesses the ability to understand and execute part-level missions and accurately segments and detects parts compared with existing interactive methods.

Information-hiding cameras: Optical concealment of object information into ordinary images.

We introduce an information-hiding camera integrated with an electronic decoder that is jointly optimized through deep learning. This system uses a diffractive optical processor, which transforms and hides input images into ordinary-looking patterns that deceive/mislead observers. This information-hiding transformation is valid for infinitely many combinations of secret messages, transformed into ordinary-looking output images through passive light-matter interactions within the diffractive processor. By processing these output patterns, an electronic decoder network accurately reconstructs the original information hidden within the deceptive output. We demonstrated our approach by designing information-hiding diffractive cameras operating under various lighting conditions and noise levels, showing their robustness. We further extended this framework to multispectral operation, allowing the concealment and decoding of multiple images at different wavelengths, performed simultaneously. The feasibility of our framework was also validated experimentally using terahertz radiation. This optical encoder-electronic decoder-based codesign provides a high speed and energy efficient information-hiding camera, offering a powerful solution for visual information security.

Implementation of Text to Image using Diffusion Model

Text-to-image generation is a transformative field in artificial intelligence, aiming to bridge the semantic gap between textual descriptions and visual representations. This presents a comprehensive approach to tackle this challenging task. Leveraging the advancements in deep learning, natural language processing (NLP), and computer vision, this proposes a cutting-edge model for generating high-fidelity images from textual prompts. Trained on a vast and varied dataset of written descriptions and related images, this model combines an image decoder and a text encoder within a hierarchical framework. To enhance realism, this incorporates attention mechanisms and fine-grained semantic parsing. The model's performance is rigorously evaluated through both quantitative metrics and qualitative human assessments. Results demonstrate its ability to produce visually compelling and contextually accurate images across various domains, from natural scenes to specific object synthesis. This further explores applications in creative content generation, design automation, and virtual environments, showcasing the potential impact of our approach. Additionally, this releases a user-friendly API, empowering developers and designers to integrate our model into their projects, and fostering innovation and creativity. Key Words: image generation model, Deep learning, Natural language processing.

STUDY OF SATELLITE IMAGES OF UKRAINE BEFORE AND AFTER MASSIVE ATTACKS ON ENERGY INFRASTRUCTURE USING FRACTAL METHODS

As world experience shows, Earth remote sensing data is increasingly being used to solve various problems. The wide application of remote sensing data is primarily due to the efficiency and visibility during the processing of data obtained from large areas. Space images from satellites in modern life have a high spatial resolution, which provides researchers and users with satisfactory initial data for solving various types of tasks. A promising direction for increasing the informativeness of space images is the use of fractal image analysis methods. The complexity of the forms of the understory surface and vegetation can be described using the fractal dimension. Characteristic values of the fractal dimension allow decoding of cosmic images. The purpose of the research is to develop a MATLAB software module that will analyze satellite images of the territory of Ukraine at different time intervals and examine their calculated fractal characteristics. Based on the received data, an analysis will be conducted and conclusions will be drawn. In the course of the research, it is necessary to determine the factors that affect the value of fractal characteristics.

AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models

Large Vision-Language Models (LVLMs) such as MiniGPT-4 and LLaVA have demonstrated the capability of understanding images and achieved remarkable performance in various visual tasks. Despite their strong abilities in recognizing common objects due to extensive training datasets, they lack specific domain knowledge and have a weaker understanding of localized details within objects, which hinders their effectiveness in the Industrial Anomaly Detection (IAD) task. On the other hand, most existing IAD methods only provide anomaly scores and necessitate the manual setting of thresholds to distinguish between normal and abnormal samples, which restricts their practical implementation. In this paper, we explore the utilization of LVLM to address the IAD problem and propose AnomalyGPT, a novel IAD approach based on LVLM. We generate training data by simulating anomalous images and producing corresponding textual descriptions for each image. We also employ an image decoder to provide fine-grained semantic and design a prompt learner to fine-tune the LVLM using prompt embeddings. Our AnomalyGPT eliminates the need for manual threshold adjustments, thus directly assesses the presence and locations of anomalies. Additionally, AnomalyGPT supports multi-turn dialogues and exhibits impressive few-shot in-context learning capabilities. With only one normal shot, AnomalyGPT achieves the state-of-the-art performance with an accuracy of 86.1%, an image-level AUC of 94.1%, and a pixel-level AUC of 95.3% on the MVTec-AD dataset.

Joint Semantic Segmentation using representations of LiDAR point clouds and camera images

LiDAR and camera are two common vision sensors used in the real world, producing complementary point cloud and image data. While multimodal data has previously been found mostly in 3D detection and tracking, we aim to study large-scale semantic segmentation by multimodal data fusion rather than only knowledge transfer or distillation. We show that fusing LiDAR features with camera features and abandoning the strict point-to-pixel hard correlation can lead to better performance. Even so, it is still difficult to make full use of multimodal data due to the spatiotemporal misalignment of sensors and uneven data distribution.. To address this issue, we propose the Joint Semantic Segmentation (JoSS), a powerful LiDAR-camera fusion solution that employs the attention mechanism to explore the potential relationships between point clouds and images. Specifically, JoSS consists of commonly used 3D and 2D backbones, and lightweight transformer decoders based on point clouds and images. A point cloud decoder adopts queries to analyze the semantics from LiDAR features, and an image decoder adaptively fuses these queries with corresponding image features. Both exploit contextual information, thus fully mining multimodal information for semantic segmentation. In addition, we propose an effective unimodal data augmentation (UDA) method that performs cross-modal contrastive learning on point clouds and images to significantly improve accuracy by augmenting the point cloud alone without the complexity of generating paired samples of both modalities. Our Joss achieves advanced results in two widely used large-scale benchmarks, i.e. SemanticKITTI and nuScenes-lidarseg.

Innovative approaches to qualitative land assessment and valuation: use of modern technologies and geoinformation x systems

The article examines modern methods and technologies that can be used to increase the accuracy of land valuation. The authors analyze the current problems in the field of rating and offer ways of integrating geoinformation systems with the aim of creating a more objective and information-rich picture of land quality. The purpose of the article. Study of the potential of using the latest technologies and geoinformation systems in determining the quality and classification of land with the intention of increasing accuracy and impartiality in the assessment of land resources. Research methods. During the research, the following methods were used: comparative analysis, the method of automated image decoding, the cartographic method and the geo-information mapping method. The results. The article emphasizes the importance of geoinformation systems in the collection, processing and analysis of data, which allows taking into account a wide range of factors when evaluating land, including erosion processes, soil fertility, ecological status and other critically important indicators. Special attention is paid to the possibilities of using remote sensing and other automated methods for data collection. Conclusions. The results of the research are of practical importance, as they can be used to optimize agricultural management, plan the use of land resources, and develop strategies for their effective use and protection. Prospects for further research may include the development of new algorithms for the analysis and processing of data obtained from satellite images, as well as the improvement of existing geoinformation models for land evaluation. Integration of the obtained models and systems into the practical aspects of land use and agricultural management is also envisaged to optimize the use of land resources and increase the efficiency of land use.

Deep Learning Design for Multiwavelength Infrared Image Sensors Based on Dielectric Freeform Metasurface

AbstractNear‐infrared multispectral imaging technology enhances target detection and recognition by distinguishing the spectral characteristics of various targets. However, traditional imaging systems heavily rely on complex optical filter designs that are often bulky and mechanically unstable, posing significant challenges for miniaturization and integration challenging. In this study, a freeform dielectric metasurface with the wavelength‐multiplexing focusing effect based on a deep learning model is designed, which can separate the mixed near‐infrared light into distinct wavelengths. To effectively modulate the complex amplitude of the transmitted light at three distinct near‐infrared wavelengths (1150, 1350, and 1550 nm), high‐index silicon freeform nanostructures supporting rich resonant modes are proposed. An inverse design model based on deep learning is utilized to generate individual freeform nanostructures pixel by pixel, satisfying the complex amplitude requirement for a multiplexed metalens design. Both the simulated and experimental results show that the wavelength‐multiplexing effect of the devices is in good agreement with the target with negligible crosstalk. Finally, a metasurface is employed to realize near‐infrared multispectral imaging, which allows for the distinct detection and decoding of images at the three target wavelengths. The proposed technology has a wide range of applications in clinical medicine, biological tissue imaging, and deep‐space exploration.

Verbal Coding of the Sun in Riddles

The paper aims at providing a comparative research of how the enigmatic spirit of riddles manifests itself when various conceptions serve to describe the Sunin Ukrainian and English traditions. The three-component cosmonym coding analysis in riddles implies the decoding of the SUN image concerning its attributive, predicative, and nominative aspects. The explicit and/or implicit manifestations of the attributive and predicative characteristics are considered to constitute the core of the SUN concept in the consciousness of Ukrainian and English language speakers. In most Ukrainian and English riddles, the description of the SUN is most frequently based on the conceptions of its round shape, unreachable location, and the main function – to shine.

Abnormal functional connectivity of the reward network is associated with social communication impairments in autism spectrum disorder: A large-scale multi-site resting-state fMRI study

BackgroundThe social motivation hypothesis proposes that the social deficits of autism spectrum disorder (ASD) are related to reward system dysfunction. However, functional connectivity (FC) patterns of the reward network in ASD have not been systematically explored yet. MethodsThe reward network was defined as eight regions of interest (ROIs) per hemisphere, including the nucleus accumbens (NAc), caudate, putamen, anterior cingulate cortex (ACC), ventromedial prefrontal cortex (vmPFC), orbitofrontal cortex (OFC), amygdala, and insula. We computed both the ROI-wise resting-state FC and seed-based whole-brain FC in 298 ASD participants and 348 typically developing (TD) controls from the Autism Brain Imaging Data Exchange I dataset. Two-sample t-tests were applied to obtain the aberrant FCs. Then, the association between aberrant FCs and clinical symptoms was assessed with Pearson's correlation or Spearman's correlation. In addition, Neurosynth Image Decoder was used to generate word clouds verifying the cognitive functions of the aberrant pathways. Furthermore, a three-way multivariate analysis of variance (MANOVA) was conducted to examine the effects of gender, subtype and age on the atypical FCs. ResultsFor the within network analysis, the left ACC showed weaker FCs with both the right amygdala and left NAc in ASD compared with TD, which were negatively correlated with the Autism Diagnostic Observation Schedule (ADOS) total scores and Social Responsiveness Scale (SRS) total scores respectively. For the whole-brain analysis, weaker FC (i.e., FC between the left vmPFC and left calcarine gyrus, and between the right vmPFC and left precuneus) accompanied by stronger FC (i.e., FC between the left caudate and right insula) were exhibited in ASD relative to TD, which were positively associated with the SRS motivation scores. Additionally, we detected the main effect of age on FC between the left vmPFC and left calcarine gyrus, of subtype on FC between the right vmPFC and left precuneus, of age and age-by-gender interaction on FC between the left caudate and right insula. ConclusionsOur findings highlight the crucial role of abnormal FC patterns of the reward network in the core social deficits of ASD, which have the potential to reveal new biomarkers for ASD.

THE ROLE OF MULTIMODAL MEANS IN EXPRESSING PROHIBITION

The article deals with the study of verbal and non-verbal means’ interaction actualising the speech acts of prohibition communicated in the spheres of publishing and advertising, as the ones aimed at engaging readers or consumers by making use of catching slogans and powerful graphic means. To define the specificity of techniques used to influence the recipients, the author singles them out into two main types: verbal (language means that represent the concept of prohibition), and visual (non-textual content of a message which consists of such aspects as graphic images, colour, and font). The study results were based on the analysis of 53 front covers of the British journal “The Economist” as well as the ads that contain the idea of prohibition. By way of using the outlined language means’ interaction, it has been found out that actualisation of prohibitive speech acts in advertising and publishing is ensured by the predominance of verbal means that contain the formula do + not + verb serving as a straightforward appeal to the recipient. The verbal message of prohibition is supplemented by colourful images, pictures, multimodal lexical units, or symbols which draw people’s attention to the encoded problem. In this case the prominence is given to colour as a dominant constituent part that not only accentuates the message visibility in the advertisement in the first place, but also helps convey its particular deep meaning, like a set of values of a country or a variety of common associations that it traditionally brings up in different cultures. Such a characteristics can be explained by a concise form of an advertisement and consequently the creators’ purpose to influence the recipient within a short period of time. Thus, the decoding of such multimodal images requires orientation in the general notions the senders refer to, as well as the time or period when it was released since extra-linguistic factors carry additional semantic loading. In view of this, the author concludes that the application of the detailed description of instrumentality to the study of multimodal means’ interaction makes it possible to present a comprehensive description of invariant patterns typical of published multimodal texts belonging to different genres.

Spatiotemporal analysis of urban sprawling using change detection: a case study of Shaki district, Azerbaijan

Introduction. The contemporary globalized world characterizes the rapid population growth, its significant concentration in cities, and an increase in the urban population. Currently, many socio-cultural, economic, environmental, and other challenges are arising in modern cities. Cities are therefore emerging from the common understanding and displaying new characteristics: reduced density, dispersed development, poor accessibility and monofunction. However, the concentration of population in cities also brings its own set of issues. The purpose of article. The purpose of the study was to identify Shaki City's urban development process, and how its land use has evolved over time. It examines the population growth in the region and the increase in the specific weight of the urban population between 2016 and 2023, analyzing the population growth trend over a 20-year period. The direction and extent of urban land use has been studied by determining the relationship between the growth rate of the urban population and the extension of the urban area, and by analyzing the changes which have occurred during the period of land use. Research methods. Statistical data has also been used for this purpose, together with data from the Azersky satellite. Machine Learning (ML), which is widely used in remote sensing systems, was applied, Support Vector Machine Learning (SVM), and image classification and processing were performed. On the basis of the obtained data, a comparative analysis of the previous and current conditions was carried out and the area of changes in the area between the classified areas was calculated. Simultaneously, the changes between categories during the use of the area and the recent changes in the direction of land use were shown. Classification performance has been assessed, user and producer accuracies have been determined and kappas have been calculated. Main findings. The increase in the population of the Shaki district led to an increase in the specific weight of the urban population and the extension of the town to the south and south-east where the population previously lived sparsely. It is mainly due to construction of new housing estates in region, as well as construction of a central clinic, an ASAN service, and a regional education division. A 'flight to the centre' was observed, resulting in noticeable changes in the land use structure between 2016 and 2022, in line with the growth rate of urbanisation and economic development. The decoding of the distribution images of the region shows that there has been an increase in the area of settlements over the six-year period. By 2016, settlements cover 22.4 per cent of the city, and by 2022, the figure rises to 39 per cent. From 34% to 32.9%, the total area of forest reserves decreased. Scientific novelty and practical value The article describes for the first time the urban sprawling and territorial transformations in Shaki district in the context of population growth by using change detection analyses. The practical value of the study is the possibility of using its algorithm and method to conduct similar studies in other cities of Azerbaijan. The results of the study are significant in the context of justifying regional measures to adapt urban expansion to population growth.

FD-CNN: A Frequency-Domain FPGA Acceleration Scheme for CNN-Based Image-Processing Applications

In the emerging edge-computing scenarios, FPGAs have been widely adopted to accelerate convolutional neural network (CNN)–based image-processing applications, such as image classification, object detection, and image segmentation, and so on. A standard image-processing pipeline first decodes the collected compressed images from Internet of Things (IoTs) to RGB data, then feeds them into CNN engines to compute the results. Previous works mainly focus on optimizing the CNN inference parts. However, we notice that on the popular ZYNQ FPGA platforms, image decoding can also become the bottleneck due to the poor performance of embedded ARM CPUs. Even with a hardware accelerator, the decoding operations still incur considerable latency. Moreover, conventional RGB-based CNNs have too few input channels at the first layer, which can hardly utilize the high parallelism of CNN engines and greatly slows down the network inference. To overcome these problems, in this article, we propose FD-CNN, a novel CNN accelerator leveraging the partial-decoding technique to accelerate CNNs directly in the frequency domain. Specifically, we omit the most time-consuming IDCT (Inverse Discrete Cosine Transform) operations of image decoding and directly feed the DCT coefficients (i.e., the frequency data) into CNNs. By this means, the image decoder can be greatly simplified. Moreover, compared to the RGB data, frequency data has a narrower input resolution but has 64× more channels. Such an input shape is more hardware friendly than RGB data and can substantially reduce the CNN inference time. We then systematically discuss the algorithm, architecture, and command set design of FD-CNN. To deal with the irregularity of different CNN applications, we propose an image-decoding-aware design-space exploration (DSE) workflow to optimize the pipeline. We further propose an early stopping strategy to tackle the time-consuming progressive JPEG decoding. Comprehensive experiments demonstrate that FD-CNN achieves, on average, 3.24×, 4.29× throughput improvement, 2.55×, 2.54× energy reduction and 2.38×, 2.58× lower latency on ZC-706 and ZCU-102 platforms, respectively, compared to the baseline image-processing pipelines.

Метод нейроморфного кодирования / восстановления изображений, заданных пуассоновскими отсчетами

The paper discusses one of the possible neuromorphic methods for processing relatively large volumes of streaming data. The method is mainly motivated by the known mechanisms of sensory perception of living systems, in particular, methods of visual perception. In this regard, the main provisions of the method are discussed in the context of problems of encoding/recovering images on the periphery of the visual system. The proposed method is focused on representing input data in the form of a stream of discrete events (samples), similar to the discharge events of retinal neurons. For these purposes, a special representation of data streams is used in the form of a controlled size sample of counts (sampling representations). Based on the specifics of the sampling representation, the generative data model is naturally formalized in the form of a system of components distributed over the field of view. Within the framework of the generative model, the optimal coding problem is formulated into the problem of searching for maximum likelihood estimates. The solution to the last problem was carried out on the basis of structuring an array of components in the form of receptive fields model, embodying universal principles (including lateral inhibition) of the neural network of the brain. The mechanism of lateral inhibition is implemented in the model in the form of an antagonistic center / environment structure of the RP. Issues of image decoding are considered in the context of restoring spatial contrasts, which partly emulates the work of the so-called simple cells of the visual cortex. It is shown that the model of coupled ON-OFF decoding allows for the restoration of sharp image details in the form of emphasizing edges, and the connection of the method with the so-called retinex theory is noted.

CIPS-3D++: End-to-End Real-Time High-Resolution 3D-Aware GANs for GAN Inversion and Stylization.

Style-based GANs achieve state-of-the-art results for generating high-quality images, but lack explicit and precise control over camera poses. Recently proposed NeRF-based GANs have made great progress towards 3D-aware image generation. However, the methods either rely on convolution operators which are not rotationally invariant, or utilize complex yet suboptimal training procedures to integrate both NeRF and CNN sub-structures, yielding un-robust, low-quality images with a large computational burden. On top of our open-source CIPS-3D framework 1https://github.com/PeterouZh/CIPS-3D, this paper presents an upgraded version called CIPS-3D++, aiming at high-robust, high-resolution and high-efficiency 3D-aware GANs. On the one hand, our basic model CIPS-3D, encapsulated in a style-based architecture, features a shallow NeRF-based 3D shape encoder as well as a deep MLP-based 2D image decoder, achieving robust image generation/editing with rotation-invariance. On the other hand, our proposed CIPS-3D++, inheriting the rotational invariance of CIPS-3D, together with geometric regularization and upsampling operations, encourages high-resolution high-quality image generation/editing with great computational efficiency. Trained on raw single-view images, without any bells and whistles, CIPS-3D++ sets new records for 3D-aware image synthesis, with an impressive FID of 3.2 on FFHQ at the 1024×1024 resolution. In the meantime, CIPS-3D++ runs efficiently and enjoys a low GPU memory footprint so that it can be trained end-to-end on high-resolution images directly, in contrast to previous alternate/progressive methods. Based on the infrastructure of CIPS-3D++, we propose a 3D-aware GAN inversion algorithm named FlipInversion, which can reconstruct the 3D object from a single-view image. We also provide a 3D-aware stylization method for real images based on CIPS-3D++ and FlipInversion. In addition, we analyze the problem of mirror symmetry suffered in training, and solve it by introducing an auxiliary discriminator for the NeRF network. Overall, CIPS-3D++ provides a strong base model that can serve as a testbed for transferring GAN-based image editing methods from 2D to 3D. Our open-source project as well as accompanying demo videos can be found online 2https://github.com/PeterouZh/CIPS-3Dplusplus.

Brain-guided manifold transferring to improve the performance of spiking neural networks in image classification.

Spiking neural networks (SNNs), as the third generation of neural networks, are based on biological models of human brain neurons. In this work, a shallow SNN plays the role of an explicit image decoder in the image classification. An LSTM-based EEG encoder is used to construct the EEG-based feature space, which is a discriminative space in viewpoint of classification accuracy by SVM. Then, the visual feature vectors extracted from SNN is mapped to the EEG-based discriminative features space by manifold transferring based on mutual k-Nearest Neighbors (Mk-NN MT). This proposed "Brain-guided system" improves the separability of the SNN-based visual feature space. In the test phase, the spike patterns extracted by SNN from the input image is mapped to LSTM-based EEG feature space, and then classified without need for the EEG signals. The SNN-based image encoder is trained by the conversion method and the results are evaluated and compared with other training methods on the challenging small ImageNet-EEG dataset. Experimental results show that the proposed transferring the manifold of the SNN-based feature space to LSTM-based EEG feature space leads to 14.25% improvement at most in the accuracy of image classification. Thus, embedding SNN in the brain-guided system which is trained on a small set, improves its performance in image classification.

Results of the Application of Direct-search Mobile Technology in the Exploration Blocks of Shakal and Halabja (Kurdistan)

Using experimental studies of mobile exploration technology, which includes modified methods of frequency-resonance processing, decoding of satellite and photographic images, vertical electron resonant scanning of the geological section, and the method of integral assessment of oil and gas content, the authors identified the potential of large prospecting blocks and licensed areas within the Shakal and Halabja blocks in Kurdistan. In the Shakal exploration area, the authors received oil responses from 3 intervals: 1) 2771-2794 m, 2) 2795.3-2815.45 m, and 3) 2834.40-2854 m in limestone sections. In Halabja exploration area, oil responses were obtained from 12 intervals: 1) 297-311.5 m, 2) 328-330 m, 3) 1190-1260 m, 4) 2018-2020 m, 5) 2059-2061 m, 6) 2132-2133 m, 7) 2192-2201 m, 8) 2249-2276 m, 9) 2307-2310 m, 10) 2317-2321 m, 11) 2326-2329 m and 12) 3310-3340 m. These results suggest that the proposed alternative exploration model can provide a more direct means of assessing the hydrocarbon potential of large exploratory areas, even before other geophysical investigations provide detailed information on possible targets.

Inferring light responses of primate retinal ganglion cells using intrinsic electrical signatures

Objective. Retinal implants are designed to stimulate retinal ganglion cells (RGCs) in a way that restores sight to individuals blinded by photoreceptor degeneration. Reproducing high-acuity vision with these devices will likely require inferring the natural light responses of diverse RGC types in the implanted retina, without being able to measure them directly. Here we demonstrate an inference approach that exploits intrinsic electrophysiological features of primate RGCs. Approach. First, ON-parasol and OFF-parasol RGC types were identified using their intrinsic electrical features in large-scale multi-electrode recordings from macaque retina. Then, the electrically inferred somatic location, inferred cell type, and average linear-nonlinear-Poisson model parameters of each cell type were used to infer a light response model for each cell. The accuracy of the cell type classification and of reproducing measured light responses with the model were evaluated. Main results. A cell-type classifier trained on 246 large-scale multi-electrode recordings from 148 retinas achieved 95% mean accuracy on 29 test retinas. In five retinas tested, the inferred models achieved an average correlation with measured firing rates of 0.49 for white noise visual stimuli and 0.50 for natural scenes stimuli, compared to 0.65 and 0.58 respectively for models fitted to recorded light responses (an upper bound). Linear decoding of natural images from predicted RGC activity in one retina showed a mean correlation of 0.55 between decoded and true images, compared to an upper bound of 0.81 using models fitted to light response data. Significance. These results suggest that inference of RGC light response properties from intrinsic features of their electrical activity may be a useful approach for high-fidelity sight restoration. The overall strategy of first inferring cell type from electrical features and then exploiting cell type to help infer natural cell function may also prove broadly useful to neural interfaces.

De-identifying transmission system using wireless channel as differential privacy noise and deep neural networks

In order to deal with the unprecedented problem of data privacy in deep-learning based computer vision technologies, we propose a de-identifying transmission system that utilizes a wireless channel as differential privacy (DP) noise and is constructed using deep neural networks. By following the Gaussian mechanism of DP, we present that the signal received at a receiver can be considered as a Gaussian mechanism. Then, we discuss the relationship between the signal-to-noise ratio of a received signal and the privacy budget of DP, and introduce how to control transmit power to achieve a specific privacy budget. The proposed system can be divided into three parts, named transmitter, wireless channel, and receiver. The transmitter and receiver are constructed by using deep-learning networks. The transmitter consists of an image encoder network based on a neural network and a power control module that controls the transmit power for adjusting the level of de-identification. The wireless channel acts as differential privacy noise, and anonymizes the transmitted image feature vector extracted from the transmitter’s image encoder network. The receiver includes a post-processing network and an image decoder network, which are also implemented by using deep neural networks. The post-processing network is proposed for high-quality decoded face images at the receiver, which maps received feature vectors perturbed by a wireless channel back into the latent space of deep-learning based image encoder and decoder networks. Finally, extensive qualitative and quantitative evaluations confirm that the proposed system can well de-identify transmitted face images only by controlling the transmit power while maintaining the usefulness of decoded face images. The proposed system shows that the Recall@1 is smaller than 5.2 and the face detection probability is larger than 90 % at SNR=2 dB. Since the de-identification process is performed by wireless channel noise, the proposed system does not require de-identification processing at a transmitter, and thus there is no burden on the transmitter to anonymize face images. Moreover, the additional advantage of this proposed system is that the level of de-identification can be controlled only by changing the transmit power. This proposed de-identification system can be utilized in a wireless image acquisition scenario where images are captured from a wireless edge, such as a CCTV camera, and then sent to a server while protecting people’s privacy.