Broadband photometry offers a time and cost effective method to reconstruct the continuum emission of celestial objects. Thus, photometric redshift estimation has supported the scientific exploitation of extragalactic multiwavelength surveys for more than twenty years. Deep fields have been the backbone of galaxy evolution studies and have brought forward a collection of various approaches in determining photometric redshifts. In the era of precision cosmology, with the upcoming Euclid and LSST surveys, very tight constraints are put on the expected performance of photometric redshift estimation using broadband photometry, thus new methods have to be developed in order to reach the required performance. We present a novel automatic method of optimizing photometric redshift performance, the classification-aided photometric redshift estimation (CPz). The main feature of CPz is the unified treatment of all classes of objects detected in extragalactic surveys: galaxies of any type (passive, starforming and starbursts), active galactic nuclei (AGN), quasi-stellar objects (QSO), stars and also includes the identification of potential photometric redshift catastrophic outliers. The method operates in three stages. First, the photometric catalog is confronted with star, galaxy and QSO model templates by means of spectral energy distribution fitting. Second, three machine-learning classifiers are used to identify 1) the probability of each source to be a star, 2) the optimal photometric redshift model library set-up for each source and 3) the probability to be a photometric redshift catastrophic outlier. Lastly, the final sample is assembled by identifying the probability thresholds to be applied on the outcome of each of the three classifiers. Hence, with the final stage we can create a sample appropriate for a given science case, for example favoring purity over completeness. We apply our method to the near-infrared VISTA public surveys, matched with optical photometry from CFHTLS, KIDS and SDSS, mid-infrared WISE photometry and ultra-violet photometry from the Galaxy Evolution Explorer (GALEX). We show that CPz offers improved photometric redshift performance for both normal galaxies and AGN without the need for extra X-ray information.