Human Silhouette Research Articles

The Untitled Images (2014)

The Untitled Images (2014) Khaled Barakeh (bio) Modern artists are facing constant challenges in terms of defining their roles—artists are no longer just themselves; they often become an artist-activist, artist-journalist, artiststoryteller, and so on. But have our times changed the rules of truth telling, storytelling through art, content choices regarding topics that could, or should, be a part of discussion in the modern art world? After a few years of repetitive images of Middle Eastern misery shown in Western media, I've noticed a certain numbness, even a cruel boredom, in viewers becoming used to seeing scenes of massacres on daily basis. At the same time, editors and international photography agencies dominate content, showing what's subjectively worth showing and hiding what's not. Those photographs are, indeed, [End Page 142] undeniably violent, but they are never more brutal than the reality itself. Editors and agencies claiming that their actions of refusal protect audiences from violence deny the existence of the people portrayed, as if ignorance could ever be pictured as protection. Their choices naturally shape viewers' understanding of the conflict, as people reflect on it according to what they see. During the past few years in Syria, this problem has escalated into a new social dynamic, creating a cruel "competition" of violent images—only those who post the most horrific and intense visual materials win the world's attention. It has even led to people staging traumatic events, replaying the horrors they didn't manage to capture on film, guided surely by a feeling of necessity to create even more intense, exaggerated visual materials to satisfy the world. It might be that we think we know history and art history, but what we really know is just a reproduction of it. This series, The Untitled Images (2014, digital C print, five photos, 21 × 30 cm each), raises the question of how the existence and nonexistence of an artwork might affect the material existence of the reality presented in a photograph. It asks, how can we shape public opinion, based on reality on the ground, without showing the reality as it is? To challenge my own critical eye, I decided to use photographs taken in different parts of Syria, joined by one recurring feature—the horror of loss. Mostly portraying adults holding their children's bodies, the photos are brutal visual evidence of an equally brutal reality. I peeled off the silhouettes like we peel dead skin off our bodies—getting rid of the unwanted, the unpleasant, the inconvenient to see. Edited in this particular manner, the photos become acceptable for the media, showing only the desired amount of pain, or a lack of thereof. There are a few different levels of violence present in those pictures. First, there is that of the regime making this horror happen; second is that of the media, cynically deciding whose pain should be displayed on the pedestal of our TV screens; and last is my own layer of violence that I, as an artist, had to apply to those photographs, when I very carefully, almost surgically, erased the skin—and therefore the people who once were individuals. The act of erasure is, in fact, a protective one: the absence of the bodies makes them more present. The real-life victims are removed and become human silhouettes—they become a symbol of any victim, anywhere in the world, at any given time. Viewers are allowed to identify themselves with a universal feeling of loss and pain, not this specific one that they believe themselves to be far away from. [End Page 143] Click for larger view View full resolution Click for larger view View full resolution [End Page 144] Click for larger view View full resolution Click for larger view View full resolution [End Page 145] Click for larger view View full resolution Khaled Barakeh Khaled Barakeh is a painter and visual artist working in various media. He graduated from the Faculty of Fine Arts in Damascus, Syria, and holds an MFA from Funen Art Academy in Odense, Denmark. His conceptual art practices focus on politics and power structures in the context of identity, culture, and history. He has exhibited at...

Face, hairstyle and clothing colour de‐identification in video sequences

The authors introduce a system for person de-identification in video data that de-identifies biometric and non-biometric features, namely faces, hairstyles and clothing colours. The authors’ system detects human faces and silhouettes in the input video and replaces the detected faces with random synthesised faces obtained using a deep convolutional generative adversarial network. Alternative hairstyles are rendered over the synthesised faces, and the human silhouette is recoloured so that skin hues are preserved and clothing hues are altered. Through the use of artificially synthesised faces that look realistic, they ensure that the de-identified image looks natural and at the same time avoid ethical and legal considerations present when using real face images as replacement faces. As they address non-biometric feature de-identification, their system offers a considerably higher level of privacy protection than commonly employed solutions that use simple image processing techniques such as blurring. Qualitative and quantitative evaluation suggests that their system produces de-identified images that look natural, at the same time being resistant to re-identification attacks.

Vision-based fall detection system for improving safety of elderly people

Recognition of human movements is very useful for several applications, such as smart rooms, interactive virtual reality systems, human detection and environment modeling. The objective of this work focuses on the detection and classification of falls based on variations in human silhouette shape, a key challenge in computer vision. Falls are a major health concern, specifically for the elderly. In this study, the detection is achieved with a multivariate exponentially weighted moving average (MEWMA) monitoring scheme, which is effective in detecting falls because it is sensitive to small changes. Unfortunately, an MEWMA statistic fails to differentiate real falls from some fall-like gestures. To remedy this limitation, a classification stage based on a support vector machine (SVM) is applied on detected sequences. To validate this methodology, two fall detection datasets have been tested: the University of Rzeszow fall detection dataset (URFD) and the fall detection dataset (FDD). The results of the MEWMA-based SVM are compared with three other classifiers: neural network (NN), naive Bayes and K-nearest neighbor (KNN). These results show the capability of the developed strategy to distinguish fall events, suggesting that it can raise an early alert in the fall incidents.

Pose recognition using convolutional neural networks on omni-directional images

Abstract Convolutional neural networks (CNNs) are used frequently in several computer vision applications. In this work, we present a methodology for pose classification of binary human silhouettes using CNNs, enhanced with image features based on Zernike moments, which are modified for fisheye images. The training set consists of synthetic images that are generated from three-dimensional (3D) human models, using the calibration model of an omni-directional camera (fisheye). Testing is performed using real images, also acquired by omni-directional cameras. Here, we employ our previously proposed geodesically corrected Zernike moments (GZMI) and confirm their merit as stand-alone descriptors of calibrated fisheye images. Subsequently, we explore the efficiency of transfer learning from the previously trained model with synthetically generated silhouettes, to the problem of real pose classification, by continuing the training of the already trained network, using a few frames of annotated real silhouettes. Furthermore, we propose an enhanced architecture that combines the calculated GZMI features of each image with the features generated at CNNs’ last convolutional layer, both feeding the first hidden layer of the traditional neural network that exists at the end of the CNN. Testing is performed using synthetically generated silhouettes as well as real ones. Results show that the proposed enhancement of CNN architecture, combined with transfer learning improves pose classification accuracy for both the synthetic and the real silhouette images.

Hemispheric asymmetries in the processing of body sides: A study with ambiguous human silhouettes

When required to indicate the perceived orientation of pictures of human silhouettes with ambiguous front/back orientation and handedness, both right- and left-handed participants perceive the figures more frequently as right-handed than as left-handed, which seems to indicate an attentional bias towards the right arm of human bodies. Given that past research exploiting the divided visual field paradigm indicated a processing advantage for contralateral body parts in both hemispheres, we tested whether human silhouettes with ambiguous handedness presented in the right visual field would be interpreted more frequently as right-handed compared with those presented in the left visual field. We confirmed the expected lateralised embodiment of ambiguous human bodies, in line with previous studies showing that right and left limbs are processed faster and/or more accurately when presented in the right and left hemifield, respectively.

Real-Time Human Body Motion Capturing System

This paper proposes a human body motion capturing system using the depth images. It consists of three processes to estimate the human pose parameters. First, we develop a pixel-based body part classifier to segment the human silhouette into different body part sub-regions and extract the primary joints. Second, we convert the distribution of the joints to the feature vector and apply the regression forest to estimate human pose parameters. Third, we apply the temporal constraints mechanism to find the best human pose parameter with the minimum estimation error. In experiments, we show that our system can operate in real-time with sufficient accuracy.

Recognizing human actions from silhouettes described with weighted distance metric and kinematics

A virtual particle random walking theory under variable velocities is presented in this paper. Under the proposed theory, the solutions of some two-dimensional Poisson equations, which are discretized by nine-point finite difference method and defined on the so-called spatial-temporal motion accumulative image stemming from human silhouettes, provide us the depth contour image for actions description. Although merely two-dimensional definition domain and concepts are related to the Poisson equations, both spatial and temporal evolution information of human actions are successfully included in the depth contour image owing to designating the travelling velocities of virtual particles according to the spatial-temporal motion accumulative image. In addition, it is worth noting that projecting three-dimensional human actions to the two-dimensional image descriptors contributes to much lower computation cost in the corresponding recognition algorithms, compared to those when using the three-dimensional spatial-temporal descriptors directly. In order to enhance the recognition accuracy, a hierarchical cascaded classifier is configured with cascading nearest neighbor classifiers, in each layer of which different kinds of shape and kinematic features of human actions are dealt with. Numerical experimental results on several public human action databases are illustrated to verify recognition performance improvements by means of the proposed algorithm.

Combined curvelets and hidden Markov models for human fall detection

Fall events detection is one of the most crucial issues in the health care of elderly people. This paper proposes an innovative approach for reliably detecting fall incidents based on human silhouette shape variation in vision monitoring. This mission is achieved by: (i) introducing the curvelet transform and area ratios for identifying human postures in images; (ii) reducing the feature vector dimension using differential evolution technique; (iii) identifying postures by a support vector machine, and (iv) adapting a hidden Markov model for classifying video sequences into non-fall and fall events. Experimental results are obtained on several "Fall Detection" datasets. For evaluation, several assessment measures are computed. These evaluation measures demonstrate the effectiveness of the proposed methodology when compared to some state-of-the-art approaches.

A Depth Video-based Human Detection and Activity Recognition using Multi-features and Embedded Hidden Markov Models for Health Care Monitoring Systems

Increase in number of elderly people who are living independently needs especial care in the form of healthcare monitoring systems. Recent advancements in depth video technologies have made human activity recognition (HAR) realizable for elderly healthcare applications. In this paper, a depth video-based novel method for HAR is presented using robust multi-features and embedded Hidden Markov Models (HMMs) to recognize daily life activities of elderly people living alone in indoor environment such as smart homes. In the proposed HAR framework, initially, depth maps are analyzed by temporal motion identification method to segment human silhouettes from noisy background and compute depth silhouette area for each activity to track human movements in a scene. Several representative features, including invariant, multi-view differentiation and spatiotemporal body joints features were fused together to explore gradient orientation change, intensity differentiation, temporal variation and local motion of specific body parts. Then, these features are processed by the dynamics of their respective class and learned, modeled, trained and recognized with specific embedded HMM having active feature values. Furthermore, we construct a new online human activity dataset by a depth sensor to evaluate the proposed features. Our experiments on three depth datasets demonstrated that the proposed multi-features are efficient and robust over the state of the art features for human action and activity recognition.

A Novel Approach for Human Action Recognition from Silhouette Images

ABSTRACTIn this work, we propose a novel human action recognition (HAR) technique for human silhouette sequence based on spatio-temporal body parts movement (STBPM) and action-code classification (ACC). STBPM feature is designed to accumulate the signature of the activity of several body parts to accomplish any action. ACC is a code-based classifier for HAR, which needs no training and the codes of any action is created by analyzing the STBPM features. The proposed approach is view independent except the top view and scale invariant. The experimental results on publicly available Weizmann, MuHVAi, and IXMAS datasets clearly show that our proposed technique outperforms the related research works in terms of accuracy in the human action detection.

Depth Images-based Human Detection, Tracking and Activity Recognition Using Spatiotemporal Features and Modified HMM

Human activity recognition using depth information is an emerging and challenging technology in computer vision due to its considerable attention by many practical applications such as smart home/office system, personal health care and 3D video games. This paper presents a novel framework of 3D human body detection, tracking and recognition from depth video sequences using spatiotemporal features and modified HMM. To detect human silhouette, raw depth data is examined to extract human silhouette by considering spatial continuity and constraints of human motion information. While, frame differentiation is used to track human movements. Features extraction mechanism consists of spatial depth shape features and temporal joints features are used to improve classification performance. Both of these features are fused together to recognize different activities using the modified hidden Markov model (M-HMM). The proposed approach is evaluated on two challenging depth video datasets. Moreover, our system has significant abilities to handle subjects body parts rotation and body parts missing which provide major contributions in human activity recognition.

Accelerometer and Camera-Based Strategy for Improved Human Fall Detection.

In this paper, we address the problem of detecting human falls using anomaly detection. Detection and classification of falls are based on accelerometric data and variations in human silhouette shape. First, we use the exponentially weighted moving average (EWMA) monitoring scheme to detect a potential fall in the accelerometric data. We used an EWMA to identify features that correspond with a particular type of fall allowing us to classify falls. Only features corresponding with detected falls were used in the classification phase. A benefit of using a subset of the original data to design classification models minimizes training time and simplifies models. Based on features corresponding to detected falls, we used the support vector machine (SVM) algorithm to distinguish between true falls and fall-like events. We apply this strategy to the publicly available fall detection databases from the university of Rzeszow's. Results indicated that our strategy accurately detected and classified fall events, suggesting its potential application to early alert mechanisms in the event of fall situations and its capability for classification of detected falls. Comparison of the classification results using the EWMA-based SVM classifier method with those achieved using three commonly used machine learning classifiers, neural network, K-nearest neighbor and naïve Bayes, proved our model superior.

Human Action Recognition from Multiple Views Based on View-Invariant Feature Descriptor Using Support Vector Machines

This paper presents a novel feature descriptor for multiview human action recognition. This descriptor employs the region-based features extracted from the human silhouette. To achieve this, the human silhouette is divided into regions in a radial fashion with the interval of a certain degree, and then region-based geometrical and Hu-moments features are obtained from each radial bin to articulate the feature descriptor. A multiclass support vector machine classifier is used for action classification. The proposed approach is quite simple and achieves state-of-the-art results without compromising the efficiency of the recognition process. Our contribution is two-fold. Firstly, our approach achieves high recognition accuracy with simple silhouette-based representation. Secondly, the average testing time for our approach is 34 frames per second, which is much higher than the existing methods and shows its suitability for real-time applications. The extensive experiments on a well-known multiview IXMAS (INRIA Xmas Motion Acquisition Sequences) dataset confirmed the superior performance of our method as compared to similar state-of-the-art methods.

FPGA Implementation of Multi-scale Pedestrian Detection in Thermal Images

Abstract In this paper an embedded vision system for human silhouette detection in thermal images is presented. As the computing platform a reprogrammable device (FPGA – Field Programmable Gate Array) is used. The detection algorithm is based on a sliding window approach, which content is compared with a probabilistic template. Moreover, detection is four scales in supported. On the used test database, the proposed method obtained 97% accuracy, with average one false detection per frame. Due to the used parallelization and pipelining real-time processing for 720 × 480 @ 50 fps and 1280 × 720 @ 50 fps video streams was achieved. The system has been practically verified in a test setup with a thermal camera.

Reasons for changes in the value of unit pressure of compression products supporting external treatment

The paper presents the basics of modelling compression products with intended values of unit pressure for body circumferences with fixed and variable radius of curvature. The derived relationships referring to the dimensions of the fabric's circumferences in a relaxed state of the product were based on Laplace law, local values of the radius of curvature, and the characteristics of stretching and relaxing (deformation) of the knitted fabric, described by experimental relation for the stress and relaxation phase for the 6th hysteresis loop, taking into account confidence intervals. The article indicates the possibilities of using 3D scanning techniques of the human body to identify the radius of curvature of various circumference of the human silhouette, for which the intended value of the unit pressure is designed, and quantitative changes in the body deformation due to compression. Classic method of modelling and design of compression products, based on a cylindrical model of the human body does not provide in each case the intended value of unit pressure, according to specific normative requirements, because it neglects the effect of different values of the radius of curvature of the body circumference and the properties of the viscoelastic knitted fabrics. The model and experimental research allowed for a quantitative and qualitative assessment of the reasons for the changes in the value of unit pressure of compression products supporting the process of external treatment.

Analysis of Gait Flow Image and Gait Gaussian Image Using Extension Neural Network for Gait Recognition

This paper proposes a new technique to recognize human gait by combining model free feature extraction approaches and a classifier. Gait flow image (GFI) and gait Gaussian image (GGI) are the two feature extraction techniques used in combination with ENN. GFI is a gait period based technique, uses optical flow features. So it directly focuses on dynamic part of human gait. GGI is another gait period based technique, computed by applying Gaussian membership function on human silhouettes. Next, ENN has been used as a classifier which combines the extension theory and neural networks. All the study has been done on CASIA-A and OU-ISIR treadmill B databases. The results derived using ENN are compared with SVM (support vector machines) and NN (Nearest neighbor) classifiers. ENN proved to give good accuracy and less iteration as compared to other traditional methods.

Unified framework for human activity recognition: An approach using spatial edge distribution and ℜ-transform

In this paper, a unified approach for the recognition of human activity using the spatial edge distribution of gradients and orientation of the human silhouettes in a video sequence is presented. The spatial edge distribution is computed on still image at different levels of resolution of sub-images to extract out the shape of the activity posture. The fuzzy trapezoidal membership function is used to extract the key frames of the activity, and the single still key image is extracted according to the histogram distance. The temporal content of the activity is extracted by the computation of orientation of the silhouettes using ℜ-transform. The ℜ-transform is applied on the binary human silhouettes, and the extraction of human silhouettes from the video sequence is done using texture based segmentation techniques. The high dimensionality of the ℜ-transform features is handled by applying Local linear embedding (LLE) dimension reduction approach. A unified model is constructed by integrating the spatial edge distribution of gradients and temporal content of the activity. The performance of the developed model is demonstrated on publicly available datasets, and the highest classification accuracy achieved on each datasets is compared with the similar state-of-the-art techniques and shows the superior performance.

Human Depth Sensors-Based Activity Recognition Using Spatiotemporal Features and Hidden Markov Model for Smart Environments

Nowadays, advancements in depth imaging technologies have made human activity recognition (HAR) reliable without attaching optical markers or any other motion sensors to human body parts. This study presents a depth imaging-based HAR system to monitor and recognize human activities. In this work, we proposed spatiotemporal features approach to detect, track, and recognize human silhouettes using a sequence of RGB-D images. Under our proposed HAR framework, the required procedure includes detection of human depth silhouettes from the raw depth image sequence, removing background noise, and tracking of human silhouettes using frame differentiation constraints of human motion information. These depth silhouettes extract the spatiotemporal features based on depth sequential history, motion identification, optical flow, and joints information. Then, these features are processed by principal component analysis for dimension reduction and better feature representation. Finally, these optimal features are trained and they recognized activity using hidden Markov model. During experimental results, we demonstrate our proposed approach on three challenging depth videos datasets including IM-DailyDepthActivity, MSRAction3D, and MSRDailyActivity3D. All experimental results show the superiority of the proposed approach over the state-of-the-art methods.

A Depth Camera-based Human Activity Recognition via Deep Learning Recurrent Neural Network for Health and Social Care Services

Human activity recognition (HAR) has become an active research topic in the fields of health and social care, since this technology offers automatic monitoring and understanding of activities of patients or residents. Depth camera-based HAR recognizes human activities using features from depth human silhouettes via conventional classifiers such as Hidden Markov Model (HMM), Conditional Random Fields etc. In this paper, we propose a new HAR system via Recurrent Neural Network (RNN) which is one of deep learning algorithms. We utilize joint angles from multiple body joints changing in time which are represented a spatiotemporal feature matrix (i.e., multiple body joint angles in time). With these derived features, we train and test our RNN for HAR. In order to evaluate our system, we have compared the performance of our RNN-based HAR against the conventional HMM- and Deep Belief Network (DBN)-based HAR using a database of Microsoft Research Cambridge-12 (MSRC-12). Our test results show that the proposed RNN-based HAR is able to recognize twelve human activities reliably and outperforms the HMM- and DBN-based HAR. We have achieved the average recognition accuracy of 99.55% for the activities. The results are 7.06% more accurate than that of the HMM-based HAR and 2.01% more accurate than that of the DBN-based HAR.

A proposed unified framework for the recognition of human activity by exploiting the characteristics of action dynamics

The aim of this paper is to present a novel integrated framework for the recognition of human actions using a spatial distribution of edge gradient (SDEG) of human pose and detailed geometric orientation of a human silhouette in a video sequence. The combined descriptor endows a wealthy feature vector dictionary having both the appearance and angular kinematics information that significantly wraps the local and global information and provides discriminative depiction for the action recognition. The SDEG is computed on a still image at different levels of resolution of sub-images, and still images of the human poses are extracted from the input video sequence using fuzzy trapezoidal membership function based on the normalized histogram distance between the contiguous segment frames. The change of geometric orientation of human silhouette with time is computed using normalized R-Transform. To validate the performance of the proposed approach, extensive experiments are conducted on five publicly available human action datasets i.e. Weizmann, KTH, Ballet Movements, Multi-view i3dPost, and IXMAS. The recognition accuracy achieved on these datasets demonstrates that the proposed approach has an abundant discriminating power of recognizing the variety of actions. Moreover, the proposed approach yields superior results when compared with similar state-of-the-art methods.