The problem statement for our project is to recognize the automatic emotions from the analysis of body movement. It has tremendous potential to bring in some changes into domains like robotics, biometric identity recognition etc. A computer can identify human emotions from the facial expressions; this may bring about changes in the way we interact with the computer. One of the challenges is to identify emotion specific features from a vast number of descriptors of human body. In this project we used feature selection frame work to accurately recognize basic emotions namely happiness, sadness, anger, depression and neutral. This project consists of two layers, the first layer, a unique combination of Analysis of variance (ANOVA) and multivariate Analysis of variance (MANOVA), to eliminate irrelevant features. The proposed model can detect complex actions like sitting, walking and few others. The proposed model achieved recognition accuracy of 90.0% during walking, 96.0% during sitting, demonstrating high accuracy and robustness of the developed method.