In the emerging era of big data astrophysics, large-scale extragalactic surveys will soon provide high-quality imaging for billions of celestial objects to answer major questions in astrophysics such as the nature of dark matter and dark energy. Precision cosmology with surveys requires accurate photometric redshift (photo-z) estimation with well-constrained uncertainties as inputs for weak lensing models to measure cosmological parameters. Machine learning methods have shown promise in optimizing the information gained from galaxy images in photo-z estimation; however, many of these methods are limited in their ability to estimate accurate uncertainties. In this work, we present one of the first applications of Bayesian convolutional neural networks (BCNNs) for photo-z estimation and uncertainties. In addition, we use conformal mapping to calibrate the photo-z uncertainties to achieve good statistical coverage. We use the public GalaxiesML data set of ∼300k galaxies from the Hyper Suprime-Cam survey containing five-band photometric images and known spectroscopic redshifts from 0 < z < 4. We find that the performance is much improved when using images compared to photometry, with the BCNN achieving 0.098 rms error, a standard outlier rate of 3.9%, a 3σ outlier rate of 4.5%, and a bias of 0.0007. The performance drops significantly beyond z > 1.5 due to the relative lack of training data beyond those redshifts. This investigation demonstrates the power of using images directly and we advocate that future photo-z analysis of large-scale surveys include galaxy images.