Chaotic dynamics provide a fast and simple means to create an excellent image cryptosystem, because it is extremely sensitive to initial conditions and system parameters, pseudorandomness, and non-periodicity. However, most chaos-based image encryption schemes are symmetric cryptographic techniques, which have been proven to be more vulnerable, compared to an asymmetric cryptosystem. This paper develops an asymmetric image cryptosystem, based on the adaptive synchronization of two different chaotic systems, namely a unified chaotic system and a cellular neural network. An adaptive controller with parameter update laws is formulated, using the Lyapunov stability theory, to asymptotically synchronize the two chaotic systems. The synchronization controller is embedded in the image cryptosystem and generates a pair of asymmetric keys, for image encryption and decryption. Using numerical simulations, three sets of experiments are conducted to evaluate the feasibility and reliability of the proposed chaos-based image cryptosystem.