Shoeprint Images Research Articles

A ruler detection method for auto-adjusting scales of shoeprint images

Digital shoeprint comparison often requires the calibration of the image resolution so that features, such as patterns in shoeprints, can be compared on the same scale. To enable scaling, a shoeprint photograph can be taken with a forensic ruler in the same frame to obtain the pixel distance between two nearby graduations. However, manually measuring the number of pixels is a time-consuming process. Additionally, the measurement process might not be conducted accurately when the image is noisy or there is distortion in the ruler. In this study, we present an automated ruler detection method for adjusting the image scale. We show that this method can accurately estimate the image scale with a mean absolute percentage error of 3%. We also conducted automated shoeprint retrieval experiments on scale-unadjusted shoeprint images to show how the automated image scaling might be used in a common forensic process. Our results from these experiments show an increase in the retrieval performance from 0.735 to 0.929 at S1 by employing this approach to adjust the shoeprint image scales.

Deep learning model for human-intuitive shoeprint reconstruction

Shoeprint reconstruction is essential in forensic science, but it is also challenging due to various inconsistencies in the patterns, textures, sizes, abrasions, etc. of shoeprints. The computational reconstruction of sharper and more complete shoeprints is conventionally conducted using handcrafted features, and it often requires human intervention. Prior studies using end-to-end machine learning approaches are limited in number and have not achieved a high level of performance. In this paper, we propose a model named ShoeRec, which employs variational autoencoder (VAE) as a component in a U-Net-like architecture to reconstruct missing regions and borders in shoeprint images. ShoeRec incorporates skip connections to preserve key patterns and employs VAE in the bottleneck to facilitate the reconstruction of desired shoeprints with the restoration of detail as perceived by humans. As a U-Net, the model skips the contextual information from the encoder to the decoder, and the compressed features in the latent space via VAE optimize the probabilistic distribution for reconstructing complete shoeprints. The reconstruction operation is automatically tuned according to the objective function, so as to reduce the structural correlation between the original and projected shoeprint and restore the absent information in the desired shoeprints. To the best of our knowledge, ShoeRec is the first deep learning infusion model that specializes in shoeprint reconstruction. The shoeprints reconstructed by ShoeRec have a close match with the originals in terms of both structures and patterns, and ShoeRec outperforms state-of-the-art generative models in human evaluation.

An automated alignment algorithm for identification of the source of footwear impressions with common class characteristics

AbstractWe introduce an algorithmic approach designed to compare similar shoeprint images, with automated alignment. Our method employs the Iterative Closest Points (ICP) algorithm to attain optimal alignment, further enhancing precision through phase‐only correlation. Utilizing diverse metrics to quantify similarity, we train a random forest model to predict the empirical probability that two impressions originate from the same shoe. Experimental evaluations using high‐quality two‐dimensional shoeprints showcase our proposed algorithm's robustness in managing dissimilarities between impressions from the same shoe, outperforming existing approaches.

Shoeprint image retrieval and crime scene shoeprint image linking by using convolutional neural network and normalized cross correlation

A shoeprint image retrieval process aims to identify and match images of shoeprints found at crime scenes with shoeprint images from a known reference database. It is a challenging problem in the forensic discipline of footwear analysis because a shoeprint found at the crime scene is often imperfect. Recovered shoeprints may be partial, distorted, left on surfaces that do not mark easily, or perhaps come from shoes that do not transfer marks easily. In this study, we present a shoeprint retrieval method by using a convolutional neural network (CNN) and normalized cross-correlation (NCC). A pre-trained CNN was used to extract features from the pre-processed shoeprint images. We then employed NCC to compute a similarity score based on the extracted image features. We achieved a retrieval accuracy of 82% in our experiments, where a “successful” retrieval means that the ground truth image was returned in the top 1% of returned images. We also extend our shoeprint retrieval method to the problem of linking shoeprints recovered from crime scenes. This new method can provide a linkage between two crime scenes if the two recovered shoeprints originated from the same shoe. This new method achieved a retrieval accuracy of 88.99% in the top 20% of returned images.

Shoeprint Image Retrieval Based on Dual Knowledge Distillation for Public Security Internet of Things

In order to implement rapid retrieval of large-scale crime scene investigation shoeprint image (SPI) in the intelligent mobile terminal of Public Security Internet of Things (PSIoT), a novel dual knowledge distillation (DKD) network is designed by fusing spatial attention (SA) distillation and feature distillation to solve its problems of limited computing and storage capacity. First, ResNet50 was modified by adding a new SA module and hash layer as the teacher model, and a light convolutional neural network (CNN) with the same SA and hash layer is designed as the student model. Then, the attention distillation loss function is constructed to distill the SA knowledge in the teacher module to the student module to improve the ability of the convolutional layer at the front end of the student network to capture the underlying visual features of the SPI. Finally, the feature distillation loss function is constructed to distill the semantic knowledge in the teacher module to the student module to improve the ability of the student module to express high-level semantics of the SPI. we compare our method on two data sets of SPID and FID-300 with other state-of-the-art methods in the SPI retrieval domain. The experimental results show that our method can improve the SPI retrieval baseline by a large margin and better than other methods.

Crime Scene Shoeprint Image Retrieval: A Review

Shoeprints performs a vital role in forensic investigations. It has been an advanced research issue in forensic science. The main purpose of shoeprint image retrieval is to acquire a ranking list of shoeprint images in a database, according to their feature similarities to the query image. In this way, a shoeprint can not only be used as an exhibit for bringing criminal charges but also to provide a clue to a case. The goal of this work is to present an overview of the existing works conducted in shoeprint image retrieval. We detail the different phases of the shoeprint retrieval task and present a summary of the state-of-the-art methods. We analyzed the difficulties and problems in this field and discussed future work directions. This review may help neophytes become involved in research easily and quickly.

Restorable-inpainting: A novel deep learning approach for shoeprint restoration

Shoeprints are important information collected at the crime scene and are of great value for forensic analysis. Shoeprints collected in real-world scenarios are normally unclear, abrasive, and lack contextual and other kinds of missing information. In this research, we apply a novel deep learning technique called restorable inpainting to repair shoeprint contours and missing parts. Existing inpainting methods aim to fill artificially occluded areas with plausible pixels, but these methods may not restore missing information for occlusions in shoeprint images. In addition, because no ground-truth shoeprints exist for training samples, inpainting occluded regions becomes challenging. In this paper, we propose DeepShoePaint, a novel deep learning approach to perform restorable inpainting by restoring synthetic information resembling desirable shoeprint images necessary for forensics. DeepShoePaint novelly adapts a probabilistic distribution borrowed from the variational autoencoder into a U-Net-like structure forming a unified architecture trained in an unsupervised fashion to restore occluded and masked regions to produce human-verifiable shoeprints. The experimental results reveal that DeepShoePaint achieves outstanding human inspection and statistical assessment results and outperforms conventional inpainting models. We believe that this study can provide valuable insights, not limited to inpainting, into restoring desirable shoeprints to automate and facilitate the forensic investigation and examination process instead of using handcrafted methods.

The impact of contextual information on decision‐making in footwear examination: An eye‐tracking study

In order to investigate whether context factors and expectations might potentially influence the decision-making of forensic footwear examiners, we collected the gaze process of experts examining the shoeprint image sets through the eye-tracking recorder. Additional to eye movement data as an objective measure, questionnaires were completed, and survey was conducted afterwards. Twenty-three qualified examiners assessed the similarity among shoe images for 22 different cases, including three sets were laterally reversed. We divided the experiment into two sessions, and then compared the examiners' performance with and without contextual information. The results showed the effects of contextual bias manipulate on both behavioral data and eye tracking data. The consensus and accuracy of examiners with contextual information were higher than those without contextual information. In the eye-tracking data, there is a significant difference between fixation counts and saccadic counts under contextual information. In addition, we found that the contextual information produced significant changes in inter-examiner consistency as measured by the Earth Mover Distance metric. However, there is no significant statistic differences in saccadic amplitude and total fixation duration of the examiners after exposure to contextual information. Our research results are instructive for understanding the cognitive process of shoeprint examination involved in real cases. In this process, stimuli related to context factors may affect decision-making and behavior. Implications for contextual effect causes are discussed.

Automatic Retrieval of Shoeprints Using Modified Multi-Block Local Binary Pattern

A shoeprint is a valuable clue found at a crime scene and plays a significant role in forensic investigations. In this paper, in order to maintain the local features of a shoeprint image and place a pattern in a block, a novel automatic method was proposed, referred to as Modified Multi-Block Local Binary Pattern (MMB-LBP). In this method, shoeprint images are divided into blocks according to two different models. The histograms of all blocks of the first and second models are separately measured and stored in the first and second feature matrices, respectively. The performance evaluations of the proposed method were carried out by comparing with state-of-the-art methods. The evaluation criteria are the successful retrieval rates obtained using the best match score at rank one and cumulative match score for the first five matches. The comparison results indicated that the proposed method performs better than other methods, in terms of retrieval of complete and incomplete shoeprints. That is, the proposed method was able to retrieve 97.63% of complete shoeprints, 96.5% of incomplete toe shoeprints, and 91.18% of incomplete heel shoeprints. Moreover, the experiments showed that the proposed method is significantly resistant to the rotation, salt and pepper noise, and Gaussian white noise distortions in comparison with the other methods.

Segmentation of the Wear Area of Planar Shoeprint Using Mean Shift

In the field of forensic science, the wear area of planar shoeprint is a useful information for analysis of criminal suspects. However, the identification of shoeprint wear areas is often subjective to the expert’s personal experience. In this paper, considering the characteristics of planar shoeprint images, a segmentation algorithm based on mean-shift is proposed to process planar shoeprint image to get the wear area. Firstly, the multiplicative intrinsic component optimization method(MICO) is employed to pre-segment the image. This method combines the level set with the offset field correction method to segment the outline of the unevenly printed shoeprint image. Secondly, we use mean shift to segment the image. Finally, we compare the actual wear area with the segmented wear area. It is found that the segmentation effect is good. By pattern recognition method, the wear area of the shoeprint is segmented to provide more objective technical support for detecting cases.

Crime Scene Shoeprint Retrieval Using Hybrid Features and Neighboring Images

Given a query shoeprint image, shoeprint retrieval aims to retrieve the most similar shoeprints available from a large set of shoeprint images. Most of the existing approaches focus on designing single low-level features to highlight the most similar aspects of shoeprints, but their retrieval precision may vary dramatically with the quality and the content of the images. Therefore, in this paper, we proposed a shoeprint retrieval method to enhance the retrieval precision from two perspectives: (i) integrate the strengths of three kinds of low-level features to yield more satisfactory retrieval results; and (ii) enhance the traditional distance-based similarity by leveraging the information embedded in the neighboring shoeprints. The experiments were conducted on a crime scene shoeprint image dataset, that is, the MUES-SR10KS2S dataset. The proposed method achieved a competitive performance, and the cumulative match score for the proposed method exceeded 92.5% in the top 2% of the dataset, which was composed of 10,096 crime scene shoeprints.

LOSGSR: Learned Opinion Score Guided Shoeprint Retrieval

As a kind of forensic evidence, a shoeprint conveys many important human characteristics, and it plays a vital role in forensic investigations. Millions of shoeprints are acquired from crime scenes, and it is a challenging task to retrieve the most similar shoeprints for a query shoeprint. Most shoeprint retrieval methods sort shoeprint images by feature similarities with respect to the query shoeprint; however, the results are not always what the investigator expects because the retrieval algorithm cannot determine what the investigator prefers based on only the content of the query shoeprint. This paper proposes a method to guide the shoeprint retrieval process to approximate what the user wants by applying learned opinion scores. Additionally, this paper improves shoeprint retrieval effectiveness by implementing the following four perspectives: 1) using the opinion scores of multiple examples to guide the results to meet the forensic experts’ expectations; 2) proposing a learning-based method to refine opinion scores, which corrects the labeled opinion scores of multiple examples and their neighbors; 3) using a manifold ranking method to propagate the opinion scores to other dataset shoeprints; and 4) introducing a coefficient matrix to prevent the tendency of the ranking scores to become low values. The experiments show that the cumulative match scores of the proposed method are more than 96.6% in the top 2% of the dataset composed of 10,096 crime scene shoeprints.

Shoe-Print Image Retrieval With Multi-Part Weighted CNN

Identifying shoe-print impressions in the scene of crime (SoC) from database images is a challenging problem in forensic science due to the complicated impressing surface, the partial absence of on-site impressions, and the huge domain gap between the query and the gallery images. The existing approaches pay much attention to feature extraction while ignoring its distinctive characteristics. In this paper, we propose a novel multi-part weighted convolutional neural network (MP-CNN) for shoe-print image retrieval. Specifically, the proposed CNN model processes images in three steps: 1) dividing the input images vertically into two parts and extracting sub-features by a parameter-shared network individually; 2) calculating the importance weight matrix of the sub-features based on the informative pixels they contained and concatenating them as the final feature, and; 3) using the triplet loss function to measure the similarity between the query and the gallery images. In addition to the proposed network, we adopt an effective strategy to enhance the quality of the images and to reduce the domain gap using the U-Net structure. The experimental evaluations demonstrate that our proposed method significantly outperforms other fine-grained cross-domain methods on SPID dataset and obtains comparative results with the state-of-the-art shoe-print retrieval methods on FID300 dataset.

Robust Shoeprint Retrieval Method Based on Local‐to‐Global Feature Matching for Real Crime Scenes

In this study, an automatic and robust crime scene shoeprint retrieval method is proposed. As most shoeprints left at crime scenes are randomly partial and noisy, crime scene shoeprint retrieval is a challenging task. To handle partial, noisy shoeprint images, we employ denoising deep belief network (DBN) to extract local features and use spatial pyramid matching (SPM) to obtain a local-to-global matching score. In this study, 536 query shoeprint images from crime scenes and a large scale database containing 34,768 shoeprint images are used to evaluate the retrieval performance. Experimental results show that the proposed method outperforms other state-of-the-art methods in terms of retrieval accuracy, feature dimension, and retrieval speed. The proposed method achieves a cumulative match score (CMS) of 65.67% at top 10 which is 5.60% higher than the second best performing method.

Automatic estimation of a scale resolution in forensic images

This paper proposes a new method for an automatic detection of a resolution of a scale or a ruler with graduation marks in the shoeprint images. The method creates a vector of the correlations estimated from the co-occurrence matrices for every row in a shoeprint image. The scale resolution is estimated from maxima in Fourier spectrum of the correlations’ vectors. The proposed method is evaluated on over 500 images taken at crime scenes and in a forensics laboratory. The experimental results indicate the possibility of applying the proposed method to automatically estimate the scale resolution in forensic images. The automatic detection of a scale resolution could be used to automatically rescale a forensic image before the printing this image in “one-to-one” scale. Furthermore, the proposed method could be used to automatically rescale images to an equal scale thus allowing to compare the images digitally.

Automatic retrieval of shoeprint images using blocked sparse representation

Shoe marks are regarded as remarkable clues which can be usually detected in crime scenes where forensic experts use them for investigating crimes and identifying the criminals. Hence, developing a robust method for matching shoeprints with one another is of critical significance which can be used for recognizing criminals. In this paper, a promising method is proposed for retrieving shoe marks by means of developing blocking sparse representation technique. In the proposed method, the queried image was divided into two blocks. Then, two sparse representations are extracted for each queried image through approximate ℓ1 minimizing method. Also, the referenced database is categorized into two parts and two separate dictionaries are developed via them. Next, using the blocks, the total errors of classes are measured by resetting the coefficients related to other classes into zero. The performance of the proposed method was evaluated via the following methods Wright’s sparse representation, extracting shoeprint image local and global features by Fourier transform, extracting shoeprint image features by Gabor transform after the image is rotated and extracting the corners of shoeprint image by Hessian and Harris’ multi-scale detectors and SIFT descriptors. Accurate detection score was obtained in terms of the ratio of the number of accurately detected images to the total test images. The results of simulations indicated that the proposed method was highly effective and efficient in retrieving shoe marks, whole shoeprints, partial toe and heel shoeprints. Furthermore, it was found that the proposed method had better performance than the other methods with which it was compared. Accurate identification rate according to cumulative match score for the first n matches was measured. That is to say, the proposed method accurately recognized 99.47% of whole shoeprints, 80.53% of partial toe shoeprints and 79.47% of partial heel shoeprints in the first rank. Also, the proposed method was compared with the other methods in terms of rotation and scale distortions. The results indicated that the proposed method was resistant to these distortions.

A manifold ranking based method using hybrid features for crime scene shoeprint retrieval

Shoeprints are frequently acquired at crime scenes and can provide vital clues for investigation. How to retrieve the most similar shoeprints available in the dataset to the highly degraded query crime scene shoeprint is a challenging work. Some existing shoeprint retrieval algorithms cannot work well with highly degraded shoeprint image on a large scale dataset, and the results are not well correlated with the forensic experts. This study proposes a manifold ranking based method using hybrid features of region and appearance to improve the retrieval performance. Manifold ranking method is introduced to narrow the well-known gap between visual features and semantic concepts. We define the ranking cost function from three perspectives: (i) the feature similarity between the query and the dataset images, (ii) the relationship between every two shoeprints in the dataset, (iii) the assigned opinion scores for multiple shoeprints left in one crime scene by the forensic expert. Experiments on the real crime scene datasets show that the cumulative match score of the proposed method is more than 93.5 % on the top 2 % of the dataset composed of 10096 crime scene shoeprints.

Shoeprint retrieval: Core point alignment for pattern comparison

Purpose: Shoeprint recognition has been widely used as forensic evidence in criminal cases. The purpose of this study is to propose a shoeprint retrieval method based on core point alignment for pattern analysis.Method: The proposed method firstly detects contour points in a black-and-white shoeprint image. These reliable contour points are selected to simulate the left and right sidelines of the shoeprint by a curve fitting method. Subsequently, the most concave points along the left and right sidelines can determine the core point of the shoeprint, thereby partitioning the shoeprint into circular regions. Next, the Zernike moments of the circular regions are calculated for pattern descriptions of each region. Finally, the Euclidean distance is measured to match the shoeprints with the same pattern.Result: The highest APR=0.726 is obtained from the first four Zernike moments with a radius of 90pixels and three baselines. The experimental results also show that the Zernike method in any order always outperforms the compared moment invariant and GLCM method. The experimental results also indicate that the core point is more stable than the gravity center in the both sets, because the standard deviation values of the core point are less than that of the gravity center.Conclusions: This study has verified that the proposed method can effectively align shoeprints for pattern comparison.

Research of shoeprint image matching based on SIFT algorithm

It researches on shoeprint image positioning and matching. Firstly, this paper introduces the algorithm of Scale- invariant feature transform (SIFT) into shoeprint matching. Then it proposes an improved matching algorithm of SIFT. Because of its good scale and illumination invariant, the algorithm can be used to go on the shoeprint matching. The secondary posi- tioning on the image will be used before the feature points are created. It makes shoeprint image in the same vertical position. After that, use the minimum Euclidean distance as the standard of shoeprint matching. According to the ratio of the minimum Euclidean distance to the second minimum Euclidean distance, the invalid points are removed. That can improve the matching efficiency. The results of experiment show that the algorithm can not only shorten the matching time, but also improve the matching accuracy with high robust.

The Use of Scale‐Invariance Feature Transform Approach to Recognize and Retrieve Incomplete Shoeprints

Shoeprints left at the crime scene provide valuable information in criminal investigation due to the distinctive patterns in the sole. Those shoeprints are often incomplete and noisy. In this study, scale-invariance feature transform is proposed and evaluated for recognition and retrieval of partial and noisy shoeprint images. The proposed method first constructs different scale spaces to detect local extrema in the underlying shoeprint images. Those local extrema are considered as useful key points in the image. Next, the features of those key points are extracted to represent their local patterns around key points. Then, the system computes the cross-correlation between the query image and each shoeprint image in the database. Experimental results show that full-size prints and prints from the toe area perform best among all shoeprints. Furthermore, this system also demonstrates its robustness against noise because there is a very slight difference in comparison between original shoeprints and noisy shoeprints.