In this study, we present a method for human activity recognition in video sequences. Human activities are often described by a holistic feature vector comprising of a set of local motion descriptors. Here, we use a novel local shape feature descriptor for human activity recognition which is an integration of moment invariants and uniform local binary patterns (MI_ULBP). This feature descriptor is passed to a binary support vector machine pattern classifier for classification of human activities. Activity recognition is achieved through probabilistic search of image feature database representing previously seen activities. Experiments are performed over four benchmark video datasets Weizmann, KTH, CASIA and Collective human activity. Visual results and quantitative comparisons with existing methods show that the proposed method gives better recognition of human activities in video sequences with varying backgrounds and viewpoints.