Autism spectrum disorders (ASD) are pervasive developmental disorders with characteristic core symptoms such as impairments in social interaction, deviance in communication, repetitive and stereotyped behavior, and impaired motor skills. Anomalies of brain structure have repeatedly been hypothesized to play a major role in the etiopathogenesis of the disorder. Our objective was to perform unbiased meta-analysis on brain structure changes as reported in the current ASD literature. We thus conducted a comprehensive search for morphometric studies by Pubmed query and literature review. We used a revised version of the activation likelihood estimation (ALE) approach for coordinate-based meta-analysis of neuroimaging results. Probabilistic cytoarchitectonic maps were applied to compare the localization of the obtained significant effects to histological areas. Each of the significant ALE clusters was analyzed separately for age effects on gray and white matter density changes. We found six significant clusters of convergence indicating disturbances in the brain structure of ASD patients, including the lateral occipital lobe, the pericentral region, the medial temporal lobe, the basal ganglia, and proximate to the right parietal operculum. Our study provides the first quantitative summary of brain structure changes reported in literature on autism spectrum disorders. In contrast to the rather small sample sizes of the original studies, our meta-analysis encompasses data of 277 ASD patients and 303 healthy controls. This unbiased summary provided evidence for consistent structural abnormalities in spite of heterogeneous diagnostic criteria and voxel-based morphometry (VBM) methodology, but also hinted at a dependency of VBM findings on the age of the patients.