Low-resolution Sets Research Articles

Tinysign: sign language recognition in low resolution settings

Tinysign: sign language recognition in low resolution settings

Fast detection of slender bodies in high density microscopy data

Computer-aided analysis of biological microscopy data has seen a massive improvement with the utilization of general-purpose deep learning techniques. Yet, in microscopy studies of multi-organism systems, the problem of collision and overlap remains challenging. This is particularly true for systems composed of slender bodies such as swimming nematodes, swimming spermatozoa, or the beating of eukaryotic or prokaryotic flagella. Here, we develop a end-to-end deep learning approach to extract precise shape trajectories of generally motile and overlapping slender bodies. Our method works in low resolution settings where feature keypoints are hard to define and detect. Detection is fast and we demonstrate the ability to track thousands of overlapping organisms simultaneously. While our approach is agnostic to area of application, we present it in the setting of and exemplify its usability on dense experiments of swimming Caenorhabditis elegans. The model training is achieved purely on synthetic data, utilizing a physics-based model for nematode motility, and we demonstrate the model’s ability to generalize from simulations to experimental videos.

Exploring Self-Supervised Vision Transformers for Gait Recognition in the Wild.

The manner of walking (gait) is a powerful biometric that is used as a unique fingerprinting method, allowing unobtrusive behavioral analytics to be performed at a distance without subject cooperation. As opposed to more traditional biometric authentication methods, gait analysis does not require explicit cooperation of the subject and can be performed in low-resolution settings, without requiring the subject's face to be unobstructed/clearly visible. Most current approaches are developed in a controlled setting, with clean, gold-standard annotated data, which powered the development of neural architectures for recognition and classification. Only recently has gait analysis ventured into using more diverse, large-scale, and realistic datasets to pretrained networks in a self-supervised manner. Self-supervised training regime enables learning diverse and robust gait representations without expensive manual human annotations. Prompted by the ubiquitous use of the transformer model in all areas of deep learning, including computer vision, in this work, we explore the use of five different vision transformer architectures directly applied to self-supervised gait recognition. We adapt and pretrain the simple ViT, CaiT, CrossFormer, Token2Token, and TwinsSVT on two different large-scale gait datasets: GREW and DenseGait. We provide extensive results for zero-shot and fine-tuning on two benchmark gait recognition datasets, CASIA-B and FVG, and explore the relationship between the amount of spatial and temporal gait information used by the visual transformer. Our results show that in designing transformer models for processing motion, using a hierarchical approach (i.e., CrossFormer models) on finer-grained movement fairs comparatively better than previous whole-skeleton approaches.

Structure-aware bottle cap art

We propose a new image processing problem that consists in approximating an input image with a set of plastic bottle caps. This problem is motivated by the appreciation caused by low resolution imaging, combined with the goal of finding a new destination for plastic caps that would be discarded and possibly damage the environment. Our solution consists in formulating an energy that measures how well a bottle cap art reproduces structures and features from the reference image and maximizing it with a simulated annealing strategy. Examples in the paper show that our method produces high quality results in this very low spatial and color resolution setting.

Disguise Resilient Face Verification

With increasing usage of face recognition algorithms, it is well established that external artifacts and makeup accessories can be applied to different facial features such as eyes, nose, mouth, and cheek, to obfuscate one’s identity or to impersonate someone else’s identity. Recognizing faces in the presence of these artifacts comprises the problem of disguised face recognition, which is one of the most arduous covariates of face recognition. The challenge becomes exacerbated when disguised faces are captured in real-time environment, with low resolution images. To address the challenge of disguised face recognition, this paper first proposes a novel multi-objective encoder-decoder network, termed as DED-Net. DED-Net attempts to learn the class variations in the feature space generated by both disguised as well non-disguised images, using a combination of Mahalanobis and Cosine distance metrics, along with Mutual Information based supervision. The DED-Net is then extended to learn from the local and global features of both disguised and non-disguised face images for efficient face recognition, and the complete framework is termed as Disguise Resilient (D-Res) framework. The efficacy of the proposed framework has been demonstrated on two real-world benchmark datasets: Disguised Faces in the Wild (DFW) 2018 and DFW2019 competition datasets. In addition, this research also emphasizes on the importance of recognizing disguised faces in low resolution settings and proposes three experimental protocols to simulate the real-world surveillance scenario. To this effect, benchmark results have been shown on seven protocols for three low resolution settings (32×32, 24×24, and 16×16) of the two DFW benchmark datasets. The results demonstrate superior performance of the D-Res framework, in comparison with benchmark algorithms. For example, an improvement of around 3% is observed on the Overall protocol of the DFW2019 dataset, where the D-Res framework achieves 96.3%. Experiments have also been performed on benchmark face verification datasets (LFW, YTF, and IJB-B), where the D-Res framework achieves improved verification accuracy.

Parallel encoding in optically multiplexed image camera for temporal resolution improvement

An optically multiplexed image camera often requires a photodetector for image sensing in conjunction with an optical multiplexer, such as a digital micromirror device or an amplitude spatial light modulator. For this type of camera, the advantages of using one photodetector, unfortunately, can be outrun by the fairly large acquisition time, consumed in the image multiplexing process. Therefore, we propose a solution to efficiently improve the image acquisition time span and to relax the computational burden in the reconstruction process. Our camera design incorporates a reflection optical multiplexer, for image encoding, in conjunction with multiple photodetectors. The working principle relies on sequential image encoding using multiple low-resolution sets of binary masks, applied synchronously. The reconstructed data sets form the segments of the final image, which are stitched together with dark boundary pixels.

Double-Sequence Frequency Synchronization for Wideband Millimeter-Wave Systems With Few-Bit ADCs

In this paper, we propose and evaluate a novel double-sequence low-resolution frequency synchronization method in millimeter-wave (mmWave) systems. In our system model, the base station uses analog beams to send the synchronization signal with infinite-resolution digital-to-analog converters. The user equipment employs a fully digital front end to detect the synchronization signal with low-resolution analog-to-digital converters (ADCs). The key ingredient of the proposed method is the custom designed synchronization sequence pairs, from which there exists an invertible function (a ratio metric) of the carrier frequency offset (CFO) to be estimated. We use numerical examples to show that the ratio metric is robust to the quantization distortion. To implement our proposed method in practice, we propose to optimize the double-sequence design parameters such that: (i) for each individual user, the impact of the quantization distortion on the CFO estimation accuracy is minimized, and (ii) the resulting frequency range of estimation can capture as many users' CFOs as possible. Numerical results reveal that our proposed algorithm can provide a flexible means to estimate CFO in a variety of low-resolution settings.

Low cost gaze estimation: knowledge-based solutions.

Eye tracking technology in low resolution scenarios is not a completely solved issue to date. The possibility of using eye tracking in a mobile gadget is a challenging objective that would permit to spread this technology to non-explored fields. In this paper, a knowledge based approach is presented to solve gaze estimation in low resolution settings. The understanding of the high resolution paradigm permits to propose alternative models to solve gaze estimation. In this manner, three models are presented: a geometrical model, an interpolation model and a compound model, as solutions for gaze estimation for remote low resolution systems. Since this work considers head position essential to improve gaze accuracy, a method for head pose estimation is also proposed. The methods are validated in an optimal framework, I2Head database, which combines head and gaze data. The experimental validation of the models demonstrates their sensitivity to image processing inaccuracies, critical in the case of the geometrical model. Static and extreme movement scenarios are analyzed showing the higher robustness of compound and geometrical models in the presence of user's displacement. Accuracy values of about 3° have been obtained, increasing to values close to 5° in extreme displacement settings, results fully comparable with the state-of-the-art.

Estimation of contrast-to-noise ratio in CT and CBCT images with varying scan settings in presence of different implant materials.

To assess the contrast-to-noise ratio (CNR) of four different types of dental implant materials in CT and cone beam CT (CBCT) images with varying scan settings. Four different types of implants: zirconium (Zr), titanium (Ti) Grade 4 and 5 and titanium-zirconium (Ti-ZrO2) alloy were placed in a 3% gelatin phantom in a cylindrical plastic container and scanned with two different CT machines (GE Medical systems and Toshiba Medical Systems) and one CBCT machine (I-CAT, Imaging Sciences International) with different voxel sizes of 0.2, 0.25, 0.3 and 0.4 mm. Images were analyzed using ImageJ software with the purpose of estimating the CNR. The CNR obtained from images acquired with CT was lower than the CBCT with all voxel sizes tested. 0.3 and 0.4 mm voxel sizes exhibited the highest CNR (p < 0.05) that gives the best image quality. Among the implant materials tested, titanium Grade 5 has the highest levels of CNR while Zirconium has the lowest (p < 0.05). The optimum protocol for radiographic follow-up in areas near implants on the I-CAT is low-resolution settings (0.3 and 0.4 mm voxel sizes) which gave the highest CNR thus image quality. In presence of Zr implants, an alternative imaging modality (i.e., MRI) may be considered to avoid low-quality images.

Diagnostic accuracy of cone beam computed tomography with different resolution settings for external root resorption

To evaluate the diagnostic accuracy of cone beam computed tomography (CBCT) with different resolution settings in detecting the simulated external root resorption defects. External root resorption defects were simulated in 51 human single rooted premolar teeth. Cavities simulating root resorption defects of 1 mm in diameter and 0.1 mm, 0.2 mm, and 0.3 mm in depth were drilled in the cervical, middle and apical thirds of lingual surfaces of the teeth. In addition to the 51 locations as controls, a total of 102 cavities were obtained in the present study. Specimens were placed in a human dry mandible and scanned by ProMax 3D and DCT PRO CBCT with different resolution settings, respectively. The three-dimensional CBCT images were evaluated by two experienced observers. The data were analyzed with receiver operating characteristics (ROC) analysis. ROC curves were generated and the area under ROC curve (Az) was employed to express the diagnostic accuracy. The diagnostic accuracy (Az value) of ProMax 3D CBCT with high, normal and low resolution settings were 0.867, 0.703 and 0.665 (P < 0.05), respectively. Defects with depths of 0.2 mm and 0.3 mm were easier to be detected than those with depths of 0.1 mm (P < 0.05). The images obtained by high resolution mode scanning had obvious advantages in detecting smaller defects (depth 0.1 mm and 0.2 mm). The DCT PRO CBCT provided 4 resolution settings including normal quality + normal resolution, normal quality + high resolution, high quality + normal resolution and high quality + high resolution. The Az values for those 4 resolution settings were 0.527, 0.725, 0.743, and 0.794 (P < 0.05), respectively. Similar to ProMax 3D CBCT, the scanning mode with high resolution played a better role in detecting the defects with depth of 0.1 mm. Except for the scanning setting mode with normal quality + normal resolution, the other three modes could well be evaluated for the defects with depth of 0.2 mm and 0.3 mm. It is concluded that the diagnostic ability for external root resorption of CBCT could be affected by resolution settings. Computer-aid imaging method can improve the CBCT diagnostic accuracy for external root resorption without increasing the radiation dose level during CBCT scanning.

Super-resolving blurry multiframe images through multiframe blind deblurring using ADMM

Multiframe super resolution (MFSR) aims to reconstruct a high resolution (HR) image from a set of low resolution (LR) images. However, the MFSR is an ill-posed problem and typically computational c...

Super-resolution for face image with an improved K-NN search strategy

Recently, neighbor embedding based face super-resolution (SR) methods have shown the ability for achieving high-quality face images, those methods are based on the assumption that the same neighborhoods are preserved in both low-resolution (LR) training set and high-resolution (HR) training set. However, due to the “one-to-many” mapping between the LR image and HR ones in practice, the neighborhood relationship of the LR patch in LR space is quite different with that of the HR counterpart, that is to say the neighborhood relationship obtained is not true. In this paper, we explore a novel and effective re-identified K-nearest neighbor (RIKNN) method to search neighbors of LR patch. Compared with other methods, our method uses the geometrical information of LR manifold and HR manifold simultaneously. In particular, it searches K-NN of LR patch in the LR space and refines the searching results by re-identifying in the HR space, thus giving rise to accurate K-NN and improved performance. A statistical analysis of the influence of the training set size and nearest neighbor number is given, experimental results on some public face databases show the superiority of our proposed scheme over state-of-the-art face hallucination approaches in terms of subjective and objective results as well as computational complexity.

Image Resolution Enhancement Using Wavelet Domain Transformation and Sparse Signal Representation

Image resolution enhancement or super-resolution (SR) problem generates a high resolution (HR) image from one or a set of low resolution (LR) images. In the past two decades, a wide variety of resolution enhancement algorithms have been proposed. These methods are confined to small scaling factors. This paper presents a novel single image resolution enhancement algorithm in wavelet domain which operates at high scaling factors. First, we perform subband decomposition on the input LR image by using discrete wavelet transform (DWT). It decomposes the LR image into different frequency subbands namely low-low (LL), low-high (LH), high-low (HL) and high-high (HH). In parallel we apply sparse representation based interpolation method on the LR image. Next, we process the three high frequency subbands in wavelet domain by applying bicubic interpolation. Finally, the interpolated high frequency subbands in addition to the sparse recovered solution are combined to produce a HR image using inverse discrete wavelet transform (IDWT). Experiments on different LR test images demonstrate that our approach produces relatively less artifacts compared to the existing methods.

A multi-frame adaptive super-resolution method using double channel and regional pixel information

Multi-frame image super-resolution (SR) aims to utilize information from a set of low resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. In this paper, we propose a novel multi-frame image super-resolution algorithm built on the regularization framework. The objective functional to be minimized for the regularization framework consists of a fidelity term and a regularization term. A new fidelity term is formed by combining L1 norm and L2 norm according to defined Residual Weight Parameters (RWP) and Channel Weight Parameters (CWP). A new regularization term which adopts Regional Adaptive Weight Coefficients (RAWC) is proposed to keep edge and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thorough experimental results show the new algorithm using both fidelity and regularization terms for SR reconstruction is effective than other methods.

Augmented Active Surface Model for the Recovery of Small Structures in CT

This paper devises an augmented active surface model for the recovery of small structures in a low resolution and high noise setting, where the role of regularization is especially important. The emphasis here is on evaluating performance using real clinical computed tomography (CT) data with comparisons made to an objective ground truth acquired using micro-CT. In this paper, we show that the application of conventional active contour methods to small objects leads to non-optimal results because of the inherent properties of the energy terms and their interactions with one another. We show that the blind use of a gradient magnitude based energy performs poorly at these object scales and that the point spread function (PSF) is a critical factor that needs to be accounted for. We propose a new model that augments the external energy with prior knowledge by incorporating the PSF and the assumption of reasonably constant underlying CT numbers.

A local structure adaptive super-resolution reconstruction method based on BTV regularization

Super-resolution (SR) image reconstruction has been one of the hottest research fields in recent years. The main idea of SR is to utilize complementary information from a set of low resolution (LR) images of the same scene to reconstruct a high-resolution image with more details. Under the framework of the regularization based SR, this paper presents a local structure adaptive BTV regularization based super-resolution reconstruction method to overcome the shortcoming of the Bilateral Total Variation (BTV) super resolution reconstruction model. The proposed method adaptively chooses prior model and regularization parameter according to the local structures. Experimental results show that the proposed method can get better reconstruction results and significantly reduces the manual workload of the regularization parameter selection.

Accurate Eye Center Location through Invariant Isocentric Patterns

Locating the center of the eyes allows for valuable information to be captured and used in a wide range of applications. Accurate eye center location can be determined using commercial eye-gaze trackers, but additional constraints and expensive hardware make these existing solutions unattractive and impossible to use on standard (i.e., visible wavelength), low-resolution images of eyes. Systems based solely on appearance are proposed in the literature, but their accuracy does not allow us to accurately locate and distinguish eye centers movements in these low-resolution settings. Our aim is to bridge this gap by locating the center of the eye within the area of the pupil on low-resolution images taken from a webcam or a similar device. The proposed method makes use of isophote properties to gain invariance to linear lighting changes (contrast and brightness), to achieve in-plane rotational invariance, and to keep low-computational costs. To further gain scale invariance, the approach is applied to a scale space pyramid. In this paper, we extensively test our approach for its robustness to changes in illumination, head pose, scale, occlusion, and eye rotation. We demonstrate that our system can achieve a significant improvement in accuracy over state-of-the-art techniques for eye center location in standard low-resolution imagery.

Aircraft classification with a low resolution infrared sensor

Existing computer simulations of aircraft infrared signature (IRS) do not account for dispersion induced by uncertainty on input parameters, such as aircraft aspect angles and meteorological conditions. As a result, they are of little use to quantify the detection performance of IR optronic systems: in this case, the scenario encompasses a lot of possible situations that must indeed be considered, but cannot be individually simulated. In this paper, we focus on low resolution infrared sensors and we propose a methodological approach for predicting simulated IRS dispersion of an aircraft, and performing a classification of different aircraft on the resulting set of low resolution infrared images. It is based on a quasi-Monte Carlo survey of the code output dispersion, and on a maximum likelihood classification taking advantage of Bayesian dense deformable template models estimation. This method is illustrated in a typical scenario, i.e., a daylight air-to-ground full-frontal attack by a generic combat aircraft flying at low altitude, over a database of 30,000 simulated aircraft images. Assuming a spatially white noise background model, classification performance is very promising, and appears to be more accurate than more classical state of the art techniques (such as kernel-based support vector classifiers).

Evaluation of an underwater solid state memory video system with application to fish abundance and diversity studies in southeast Australia

This paper describes a solid state memory (SSM) video camera system designed for extended deployment as a remote underwater video device with applications for measuring fish abundance and diversity. The video camera was evaluated in a series of lab-based trials to establish performance parameters (power consumption, data accumulation rates and maximum continuous recording time). The system performed reliably with a mean maximum continuous recording time of 13.3 (±0.3) h. Field trials were carried out to (a) compare results of video parameters, number of species identified (Nsp), maximum number of each species (MaxN) and time at first appearance of each species (TFAP) to a conventional standard definition video system and (b) examine the effect that a reduction in image resolution has on these parameters. Results indicated no significant differences between the standard definition camera and the test system at the highest resolution stetting (640 × 480). However, the total number of species identified by the reader was significantly less at the low resolution setting (176 × 144) and a significant reduction in relative abundance (MaxN) and TFAP were identified for both the mid (320 × 240) and lowest resolution (176 × 144) settings. Results indicate the solid state camera system is useful for extending recording time above that of standard definition cameras. However, there is limited potential to extend deployment times of BRUV systems via a reduction in video resolution without significant compromises to the accuracy and precision of the key indicators for diversity and abundance of fishes.

Joint Super-Resolution and segmentation from a set of Low Resolution images using a Bayesian approach with a Gauss-Markov-Potts Prior

This paper addresses the problem of creating a Super-Resolution (SR) image from a set of Low Resolution (LR) images. SR image reconstruction can be viewed as a three-task process: registration or motion estimation, Point Spread Function (PSF) estimation and High Resolution (HR) image reconstruction. In the current work, we propose a new method based on the Bayesian estimation with a Gauss-Markov-Potts Prior Model (GMPPM) where the main objective is to get a new HR image from a set of severely blurred, noisy, rotated and shifted LR images. As a by-product of our prior model, we obtain jointly an SR image and an optimal segmentation of it. The proposed algorithm is unsupervised. A comparison of the performances of the proposed method with some classical and recent SR methods is provided in simulation.