Human Image Data Research Articles

Multivariate Cluster Point Process to Quantify and Explore Multi-Entity Configurations: Application to Biofilm Image Data.

Clusters of similar or dissimilar objects are encountered in many fields. Frequently used approaches treat each cluster's central object as latent. Yet, often objects of one or more types cluster around objects of another type. Such arrangements are common in biomedical images of cells, in which nearby cell types likely interact. Quantifying spatial relationships may elucidate biological mechanisms. Parent-offspring statistical frameworks can be usefully applied even when central objects ("parents") differ from peripheral ones ("offspring"). We propose the novel multivariate cluster point process (MCPP) to quantify multi-object (e.g., multi-cellular) arrangements. Unlike commonly used approaches, the MCPP exploits locations of the central parent object in clusters. It accounts for possibly multilayered, multivariate clustering. The model formulation requires specification of which object types function as cluster centers and which reside peripherally. If such information is unknown, the relative roles of object types may be explored by comparing fit of different models via the deviance information criterion (DIC). In simulated data, we compared a series of models' DIC; the MCPP correctly identified simulated relationships. It also produced more accurate and precise parameter estimates than the classical univariate Neyman-Scott process model. We also used the MCPP to quantify proposed configurations and explore new ones in human dental plaque biofilm image data. MCPP models quantified simultaneous clustering of Streptococcus and Porphyromonas around Corynebacterium and of Pasteurellaceae around Streptococcus and successfully captured hypothesized structures for all taxa. Further exploration suggested the presence of clustering between Fusobacterium and Leptotrichia, a previously unreported relationship.

Classification of human protein cell images using deep neural networks

Previously, single classification models were mainly studied to classify human protein cell images, i.e., to identify a certain protein based on a set of different cells. However, a classifier can identify only one protein, in fact, a single cell usually consists of multiple proteins, and the proteins are not completely independent of each other. In this paper, we build a human protein cell classification model by multi-label learning. The logical relationship and distribution characteristics among the labels are analyzed to determine the different proteins contained in a set of different cells (i.e., containing multiple elements in the output space). In this paper, using human protein image data, we conducted comparison experiments on pre-trained Xception and InceptionResnet V2 to optimize the two models in terms of data augmentation, channel settings, and model structure. The results show that the Optimized InceptionResnet V2 model achieves high performance in the classification task. The final accuracy of the Optimized InceptionResnet V2 model we obtained reached 96.1%, which is a 2.82% improvement relative to that before the optimized model.

Detecting depression tendency based on deep learning and multi-sources data

In the past decade, cases of depression have been increasingly reported. While early detection offers excellent advantages in reducing these cases, it still faces several challenges. Firstly, the traditional diagnosis of depression is time-consuming and expensive. Secondly, detecting depression by deep learning that solely relies on single source data leads to low prediction accuracy. Thirdly, there is a lack of established linkage datasets based on individuals’ multi-source information for detecting depression tendencies. To address these limitations, the study collected multi-source linkage datasets, including social network text data, human gait image data, and human gait key point data to establish three deep learning models: the social network model based on BERT, the gait image model based on 3D convolutional, and the gait key point model based on LSTM, for predicting depression tendencies. An integrated model was then created by adopting the Majority Voting method for the ensemble learning of these three models. The experimental results demonstrated that the accuracy of the integrated model (91.51 %) was significantly improved compared to the three models mentioned before and related models based on single source data, highlighting the necessity and effectiveness of the proposed method based on multi-sources data. Finally, this study offers practical implications for the development of self-testing Apps as well as the improvement of work efficiency for psychological clinicians.

Image Processing

Image processing entails altering an image's composition to enhance its graphical content for human interpretation and autonomous machine perception. Digital image processing is a subset of the electronic domain in which an image is transformed into a collection of tin y numbers, or pixels, that reflect a physical property, like s cene radiance, are stored in a digital memory, and are then processed by a computer or other digital hardware. Two key application areas have sparked interest in digital i mage processing techniques: improving pictorial information for human interpretation and processing image data for storage, transmission, and representation for autonomous machine perception. Boundaries are described by edges, and as edge detection i s one of the most challenging image processing tasks, it is an issue of fundamental significance .

Object Detection and Monitor System for Building Security Based on Internet of Things (IoT) Using Illumination Invariant Face Recognition

Theft and intrusion are crimes that often occur in neighborhoods when there is opportunity or negligence by owners and security personnel. Many studies have been carried out to improve environmental security by applying cameras as a surveillance medium. However, the camera is not optimal in detecting objects when the lighting conditions are lacking. Therefore, in this study, a monitoring and object detection system was built by applying the Illumination Invariant model. This model is used to improve the appearance of the image from light and shadow reflections. The process of detecting and identifying objects is done by using human facial features (face detection) captured by the camera. The camera used is a Logitec C270 Webcam 720p which is connected via a USB port on the Raspberry Pi 4. The Raspberry Pi 4 processes human face image data and sends the processing results to a MySQL database using the HTTP protocol. Data transmission is done using the Python Flask web framework. The system was successfully run 100% by using black box testing of all functional requirements. Tests on the object detection feature were carried out based on different lighting conditions 15 times by comparing the original image and the results of the Illumination Invariant implementation. Based on the test results obtained object detection accuracy of 86.7%.

Aortic dissection detection and thrombus structure quantification using volumetric ultrasound, histology, and scanning electron microscopy

Aortic dissection occurs when a weakened portion of the intima tears, and a separation of layers propagates along the aortic wall to form a false lumen filled with active blood flow or intramural thrombus. The unpredictable nature of aortic dissection formation and need for immediate intervention leaves limited serial human image data to study the formation and morphological changes that follow dissection. We used volumetric ultrasound examination, histology, and scanning electron microscopy (SEM) to examine intramural thrombi at well-defined timepoints after dissection occurs in apolipoprotein E-deficient mice infused with angiotensin II (n = 71). Stratification of red blood cell (RBC) morphologies (biconcave, intermediate biconcave, intermediate polyhedrocyte, and polyhedrocyte) in the thrombi with scanning electron microscopy (n = 5) was used to determine degree of thrombus deposition/contraction. Very few biconcave RBCs (1.2 ± 0.6%) were in the thrombi, and greater amounts of intermediate biconcave RBCs (25.8 ± 6.7%) were located in the descending thoracic portion of the dissection while more polyhedrocytes (14.6 ± 5.1%) and fibrin (42.3 ± 4.5%; P < .05) were found in the distal suprarenal aorta. Thrombus deposition likely plays some role in patient outcomes, and this multimodality technique can help investigate thrombus deposition and characteristics in experimental animal models and human tissue samples.

FP-Net: Frontal Pose Estimation Based on Single Human Body Image withArbitrary View

Accurate extraction of the frontal pose of the human body can assist behavior recognition, image generation, and virtually try-on work. However, side images and back images have problems, such as self-occlusion of the human body and invisible key points, which make the extraction of the frontal pose very difficult. Therefore, a frontal pose estimation network FP-Net (frontal pose network) based on single human body image at arbitrary view is designed and implemented. First, a multi-view human body image data set is produced to provide data support for model design. Second, in order to improve the accuracy of model prediction results, a regression module based on Anchor pose and a feature fusion module based on 3D pose are designed. Finally, FP-Net realizes the frontal pose extraction of human body images from arbitrary view. The PCK evaluation index is used for ablation experiments on the BJUT Taichi and CMU Panoptic datasets. The results show that proposed method effectively improves the accuracy of the frontal pose estimation of the human body.

Combined YOLOv5 and HRNet for High Accuracy 2D Keypoint and Human Pose Estimation

Abstract Two-dimensional human pose estimation has been widely applied in real-world applications such as sports analysis, medical fall detection, human-robot interaction, with many positive results obtained utilizing Convolutional Neural Networks (CNNs). Li et al. at CVPR 2020 proposed a study in which they achieved high accuracy in estimating 2D keypoints estimation/2D human pose estimation. However, the study performed estimation only on the cropped human image data. In this research, we propose a method for automatically detecting and estimating human poses in photos using a combination of YOLOv5 + CC (Contextual Constraints) and HRNet. Our approach inherits the speed of the YOLOv5 for detecting humans and the efficiency of the HRNet for estimating 2D keypoints/2D human pose on the images. We also performed human marking on the images by bounding boxes of the Human 3.6M dataset (Protocol #1) for human detection evaluation. Our approach obtained high detection results in the image and the processing time is 55 FPS on the Human 3.6M dataset (Protocol #1). The mean error distance is 5.14 pixels on the full size of the image (1000 × 1002). In particular, the average results of 2D human pose estimation/2D keypoints estimation are 94.8% of PCK and 99.2% of PDJ@0.4 (head joint). The results are available.

Improved SSD network for fast concealed object detection and recognition in passive terahertz security images

With the strengthening of global anti-terrorist measures, it is increasingly important to conduct security checks in public places to detect concealed objects carried by the human body. Research in recent years has shown that deep learning is helpful for detecting concealed objects in passive terahertz images. However, previous studies have failed to achieve superior accuracy and performance for real-time labeling. Our research aims to propose a novel method for accurate and real-time detection of concealed objects in terahertz images. To reach this goal we trained and tested a promising detector based on deep residual networks using human image data collected by passive terahertz devices. Specifically, we replaced the backbone network of the SSD (Single Shot MultiBox Detector) algorithm with a more representative residual network to reduce the difficulty of network training. Aiming at the problems of repeated detection and missed detection of small targets, a feature fusion-based terahertz image target detection algorithm was proposed. Furthermore, we introduced a hybrid attention mechanism in SSD to improve the algorithm’s ability to acquire object details and location information. Finally, the Focal Loss function was introduced to improve the robustness of the model. Experimental results show that the accuracy of the SSD algorithm is improved from 95.04 to 99.92%. Compared with other current mainstream models, such as Faster RCNN, YOLO, and RetinaNet, the proposed method can maintain high detection accuracy at a faster speed. This proposed method based on SSD achieves a mean average precision of 99.92%, an F1 score of 0.98, and a prediction speed of 17 FPS on the validation subset. This proposed method based on SSD-ResNet-50 can provide a technical reference for the application and development of deep learning technology in terahertz smart security systems. In the future, it can be widely used in some public scenarios with real-time security inspection requirements.

Kinship identification using age transformation and Siamese network.

Facial images are used for kinship verification. Traditional convolutional neural networks and transfer learning-based approaches are presently used for kinship identification. The transfer-learning approach is useful in many fields. However, it does not perform well in the identification of humans’ kinship because transfer-learning models are trained on a different type of data that is significantly different as compared to human face image data, a technique which may be able for kinship identification by comparing images of parents and their children with transformed age instead of comparing their actual images is required. In this article, a technique for kinship identification using a Siamese neural network and age transformation algorithm is proposed. The results are satisfactory as an overall accuracy of 76.38% has been achieved. Further work can be carried out to improve the accuracy by improving the Life Span Age Transformation (LAT) algorithm for kinship identification using facial images.

Intelligent Recognition Method of Athlete Wrong Movement Based on Image Vision

Current methods of human body movement recognition neglect the depth denoising and edge restoration of movement image, which leads to great error in athletes’ wrong movement recognition and poor application intelligence. Therefore, an intelligent recognition method based on image vision for sports athletes’ wrong actions is proposed. The basic principle, structure, and 3D application of computer image vision technology are defined. Capturing the human body image and point cloud data, the three-dimensional dynamic model of sports athletes action is constructed. The color camera including CCD sensor and CMOS sensor is selected to collect the wrong movement image of athlete and provide image data for the recognition of wrong movement. Wavelet transform coefficient and quantization matrix threshold are introduced to denoise the wrong motion images of athletes. Based on this, the feature of sports athlete’s motion contour image is extracted in spatial frequency domain, and the edge of the image is further recovered by Canny operator. Experimental results show that the proposed method can accurately identify the wrong movements of athletes, and there is no redundancy in the recognition results. Image denoising effect is good and less time-consuming and can provide a reliable basis for related fields.

An Integrated Method for Large Deformable Registration of Brain Images.

Large deformable registration of brain images is essential for a variety of clinical imaging applications. State-of-the-art diffeomorphic registration methods, such as large deformation diffeomorphic mapping (LDDMM), have high computational complexity and often require pre-processing to account for large, global displacements or rotations. In this paper, we present an integrated method that fuses landmark-based thin-plate splines (TPS), patch-based B-spline and partial differential equation (PDE) based registrations synergistically to achieve improved accuracy and efficiency for large deformable registration of brain image. Landmark-based TPS and patch-based B-spline were used for global affine transformation followed by deformable registration using LDDMM. The anatomical discrepancies between the source and target images were significantly reduced after TPS and B-spline based registration. As a result, the PDE based deformable registration could be done efficiently and effectively. The performance of the proposed method has been evaluated using simulation and real human brain image data, which provided more accurate registration than spline or PDE-based methods. Moreover, the computational efficiency of our method was significantly better than PDE-based methods. The proposed method may be useful for handling large deformable registration of brain images in various brain imaging applications.

VR/AR Technology in Human Anatomy Teaching and Operation Training.

AR/VR technology can fuse the clinical imaging data and information to build an anatomical environment combining virtual and real, which is helpful to improve the interest of teaching and the learning initiative of medical students, and then improve the effect of clinical teaching. This paper studies the application and learning effect of the VR/AR system in human anatomy surgery teaching. This paper first shows the learning environment and platform of the VR/AR system, then explains the interface and operation of the system, and evaluates the teaching situation. This paper takes the VR/AR operation simulation system of an Irish company as an example and evaluates the learning effect of 41 students in our hospital. Research shows that the introduction of the feature reweighting module in the VR/AR surgery simulation system improves the accuracy of bone structure segmentation (IOU value increases from 79.62% to 83.56%). For real human ultrasound image data, the IOU value increases from 80.21% to 82.23% after the feature reweighting module is introduced. Therefore, the dense convolution module and feature reweighting module improve the learning ability of the network for bone structure features in ultrasound images from two aspects of feature connection and importance understanding and effectively improve the performance of bone structure segmentation.

Measurement of PET Quantitative Bias In Vivo.

Quantitative imaging biomarkers are widely used in PET for both research and clinical applications, yet bias in the underlying image data has not been well characterized. In the absence of a readily available reference standard for in vivo quantification, bias in PET images has been inferred using physical phantoms, even though arrangements of this sort provide only a poor approximation of the imaging environment in real patient examinations. In this study, we used data acquired from patient volunteers to assess PET quantitative bias in vivo. Image-derived radioactivity concentrations in the descending aorta were compared with blood samples counted on a calibrated γ-counter. Methods: Ten patients with prostate cancer were studied using 2-(3-(1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl)-ureido)-pentanedioic acid PET/CT. For each patient, 3 whole-body PET/CT image series were acquired after a single administration of the radiotracer: shortly after injection as well as approximately 1 and 4 h later. Venous blood samples were obtained at 8 time points over an 8-h period, and whole blood was counted on a NaI γ-counter. A 10-mm-diameter, 20-mm-long cylindric volume of interest was positioned in the descending thoracic aorta to estimate the PET-derived radioactivity concentration in blood. A triexponential function was fit to the γ-counter blood data and used to estimate the radioactivity concentration at the time of each PET acquisition. Results: The PET-derived and γ-counter-derived radioactivity concentrations were linearly related, with an R2 of 0.985, over a range of relevant radioactivity concentrations. The mean difference between the PET and γ-counter data was 4.8% ± 8.6%, with the PET measurements tending to be greater. Conclusion: Human image data acquired on a conventional whole-body PET/CT system with a typical clinical protocol differed by an average of around 5% from blood samples counted on a calibrated γ-counter. This bias may be partly attributable to residual uncorrected scatter or attenuation correction error. These data offer an opportunity for the assessment of PET bias in vivo and provide additional support for the use of quantitative imaging biomarkers.

Automated sperm morphology analysis approach using a directional masking technique

Sperm Morphology is the key step in the assessment of sperm quality. Due to the effect of misleading human factors in manual assessments, computer-based techniques should be employed in the analysis. In this study, a computation framework including multi-stage cascade connected preprocessing techniques, region based descriptor features, and non-linear kernel SVM based learning is proposed for the classification of any stained sperm images for the assessment of the morphology. The proposed framework was evaluated on two sperm morphology datasets: the Human Sperm Head Morphology dataset (HuSHeM) and Sperm Morphology Image Data Set (SMIDS). The results indicate that cascading the preprocessing techniques used in the proposed framework, such as wavelet based local adaptive de-noising, modified overlapping group shrinkage, image gradient, and automatic directional masking, increased the classification accuracy by 10% and 5% for the HuSHeM and SMIDS, respectively. The proposed framework results in better overall accuracy than most state-of-the-art methods, while having significant advantages, such as eliminating the exhaustive manual orientation and cropping operations of the competitors with reasonable rates of consumption of time and source.

Wavelet-based computationally-efficient computer-aided characterization of liver steatosis using conventional B-mode ultrasound images

Hepatic steatosis occurs when lipids accumulate in the liver leading to steatohepatitis, which can evolve into cirrhosis and consequently may end with hepatocellular carcinoma. Several automatic classification algorithms have been proposed to detect liver diseases. However, some algorithms are manufacturer-dependent, while others require extensive calculations and consequently prolonged computational time. This may limit the development of real-time and manufacturer-independent computer-aided detection of liver steatosis. This work demonstrates the feasibility of a computationally-efficient and manufacturer-independent wavelet-based computer-aided liver steatosis detection system using conventional B-mode ultrasound (US) imaging. Seven features were extracted from the approximation part of the second-level wavelet packet transform (WPT) of US images. The proposed technique was tested on two datasets of ex-vivo mice livers with and without gelatin embedding, in addition to a third dataset of in-vivo human livers acquired using two different US machines. Using the gelatin-embedded mice liver dataset, the technique exhibited 98.8% accuracy, 97.8% sensitivity, and 100% specificity, and the frame classification time was reduced from0.4814s using original US images to 0.1444s after WPT preprocessing. When the other mice liver dataset was used, the technique showed 85.74% accuracy, 84.4% sensitivity, and 88.5% specificity, and the frame classification time was reduced from 0.5612s to 0.2903s. Using human liver image data, the best classifier exhibited 92.5% accuracy, 93.0% sensitivity, 91.0% specificity, and the classification time was reduced from 0.660s to 0.146s. This technique can be useful for developing computationally-efficient and manufacturer-independent noninvasive CAD systems for fatty liver detection.

Adaptive clustering algorithm based on kNN and density

Although many clustering algorithms have been proposed, they all have various limitations. Existing clustering algorithms usually require the user to set the appropriate threshold parameters and those parameters are usually not adaptive. In this paper, a new clustering method is proposed. Firstly, the k nearest neighbors of all samples is calculated, and then a density method based on kNN is used to complete the clustering process. In order to achieve this goal, a statistics alpha is proposed for measuring the degree of denseness or sparseness of distribution of the entire samples, and a statistical model is proposed to calculate the local parameters, beta and gamma. That is, the category of each sample is determined by the combination of kNN with density. That is to say, the total number of samples determines an appropriate k value to obtain the ordered kNN of every sample, and then the radius of surrounding region (SR) is calculated by 5NN of all samples. Furthermore, the density of every sample is calculated. In the whole process of clustering, the global threshold is determined by the density distribution of all samples, and then the local threshold is self-adaptive. All sample density is sorted to search automatically for clusters from the highest point of density of the distribution of all samples. The algorithm can not only discover clusters of arbitrary shapes and automatically remove noise and outliers, but also it can find clusters with different densities and those with internal density variation. The results of experiments on the synthetic data and human face image data demonstrate that the algorithm is effective. The code is available under the following URL: https://github.com/bing-shi/Acnd.

Convolution of 3D Gaussian surfaces for volumetric intensity inhomogeneity estimation and correction in 3D brain MR image data

Magnetic resonance (MR) imaging technique has become indispensable in image‐guided diagnosis and clinical research. However, present MR image acquisition leads to a slow varying intensity inhomogeneity (IIH) in MR image data. This study presents a novel technique based on convolution of three‐dimensional (3D) Gaussian surfaces, which is denoted as ‘Co3DGS’, for volumetric IIH estimation and correction for 3D brain MR image data. A 3D Gaussian surface is approximated using local voxel gradients on each tissue volume corresponding to grey matter, white matter and cerebrospinal fluid of the 3D brain MR image data and then convolved to partially estimate the IIH, which is subsequently removed from the image data. The above processes are repeated until there is no such significant change in the voxel gradients. The Co3DGS technique has been tested on both synthetic and in‐vivo human 3D brain MR image data of different pulse sequences. The empirical results both in qualitatively and quantitatively, which include coefficient of joint variation, index of variation, index of joint variation, index of class separability and root mean square error, collectively demonstrate that the Co3DGS efficiently estimates and removes the IIH from the 3D brain MR image data and stands superior to some state‐of‐the‐art methods.

Re-identification by neighborhood structure metric learning

Re-identifying persons of interest among distributed cameras remains a challenge in current academia and industry. Because feature designing is inevitably subject to handcrafting subjectivity and real-scenario complexity, learning a discriminative metric has gained increasing attention to date. Although metric learning has achieved inspiring results, the research progress seems to slow down far before the performance is satisfactory. The difficulty mainly comes from the variability and sparsity of human image data, which impairs traditional metric learning models based on point-wise dissimilarity. In consideration of the neighborhood structure manifold which exploits the relative relationship between the concerned samples and their neighbors in the feature space, we propose a novel method, “Neighborhood Structure Metric Learning”, to learn discriminative dissimilarities on such manifold by adapting the codomain metrics of its charts. Experiments on widely-used benchmarks have demonstrated the advantage of this method in terms of effectiveness, robustness, efficiency, stability, and generalizability.

An iterative closest point approach for the registration of volumetric human retina image data obtained by optical coherence tomography

This paper introduces an improved approach for the volume data registration of human retina. Volume data registration refers to calculating out a near-optimal transformation between two volumes with overlapping region and stitching them together. Iterative closest point (ICP) algorithm is a registration method that deals with registration between points. Classical ICP is time consuming and often traps in local minimum when the overlapping region is not big enough. Optical Coherence Tomography (OCT) volume data are several separate, partially overlapping tiles. To stitch them together is a technology in computer aided diagnosis. In this paper, a new 3D registration algorithm based on improved ICP is presented. First, the Canny edge detector is applied to generate the point cloud set of OCT images. After the detection step, an initial registration method based on the feature points of the point cloud is proposed to determine an initial transformation matrix by using singular value decomposition (SVD) method. Then, an improved ICP method is presented to accomplish fine registration. Corresponding point in the point cloud is weighted to reduce the iteration times of ICP algorithm. Finally, M-estimation is used as the objective function to decrease the impact of outliers. This registration algorithm is used to process human retinal OCT volume pairs that contain an overlapping region of 75ź?ź500ź?ź375 voxels approximately. Then a comparative experiment is conducted on some public-available datasets. The experimental results show that the proposed method outperforms the classical method.