This study proposes a novel morphological building index (MBI) for automatic building extraction from high-resolution remotely sensed imagery. The basic idea of MBI is to build a relationship between the implicit characteristics of buildings (e.g., brightness, size, and contrast) and the properties of morphological operators (e.g., reconstruction, granulometry, and directionality). Buildings are extracted by performing a threshold on the MBI feature image. Subsequently, the shape features, such as area and length-width ratio, are used to refine the binary building map. In order to validate the proposed algorithm, a comparative study was performed between MBI, a recently developed texture-derived built-up presence index (PanTex), and the widely used object-based approach. Experiments were conducted on a multispectral GeoEye-1 image, covering a study area of 5.5 km by 5.3 km in Hongshan district of Wuhan, central China. In experiments, MBI achieved satisfactory results and outperformed other algorithms in terms of both accuracies and visual inspection. The effects of parameters of MBI were also analyzed in detail, including directions, sizes and the binaryzation threshold.