Adhesion molecules hold cells together but also couple cell membranes to a contractile actomyosin network, which limits the expansion of cell contacts. Despite their fundamental role in tissue morphogenesis and tissue homeostasis, how adhesion molecules control cell shapes and cell patterns in tissues remains unclear. Here we address this question in vivo using the Drosophila eye. We show that cone cell shapes depend little on adhesion bonds and mostly on contractile forces. However, N-cadherin has an indirect control on cell shape. At homotypic contacts, junctional N-cadherin bonds downregulate Myosin-II contractility. At heterotypic contacts with E-cadherin, unbound N-cadherin induces an asymmetric accumulation of Myosin-II, which leads to a highly contractile cell interface. Such differential regulation of contractility is essential for morphogenesis as loss of N-cadherin disrupts cell rearrangements. Our results establish a quantitative link between adhesion and contractility and reveal an unprecedented role of N-cadherin on cell shapes and cell arrangements.