The breast and prostate glands are the two major organs that are highly dependent on the gonadal steroid hormones for their development and homeostasis. The cancers of these organs also show a large dependence on steroid hormones and have formed the basis of endocrine therapy. Estrogen deprivation by oophorectomy has been in active practice since the 1970s, and androgen deprivation therapy for prostate cancer was a major breakthrough in medicine in 1941. Since then, several improvisations have happened in these modes of therapy. However, the development of resistance to this deprivation and the emergence of hormone independence are major problems in both cancers. The lessons learned from rodent models have made it clear that the male hormone has a role in females and vice versa. Also, the metabolic products of these hormones may have unintentional effects including proliferative conditions in both sexes. Hence, administering estrogen as a method of chemical castration in males and administering DHT in females may not be the ideal scenario. It would be important to consider the status of the opposite sex hormone signaling and its effects and come up with a combinatorial regime to strike a balance between androgen and estrogen signaling. This review summarizes the current understanding and developments in this field in the context of prostate cancer.