Anemia is a universal public health issue, which occurs as the result of a reduction in red blood cells. This disease is common among children in Africa and other developing countries. If not treated early, children may suffer long-term consequences such as impairment in social, emotional, and cognitive functioning. Early detection of anemia in children is highly desirable for effective treatment measures. While there has been research into the development of computer-aided diagnosis (CAD) systems for anemia diagnosis, a significant proportion of these studies encountered limitations when working with limited datasets.To overcome the existing issues, this paper proposes a large dataset, named CP-AnemiC, comprising 710 individuals (range of age, 6–59 months), gathered from several hospitals in Ghana. The conjunctiva image-based dataset is supported with Hb levels (g/dL) annotations for accurate diagnosis of anemia. A joint deep neural network is developed that simultaneously classifies anemia and estimates hemoglobin levels (g/dL) based on the conjunctival pallor images. This paper conducts a comprehensive experiment on the CP-AnemiC dataset. The experimental results demonstrate the efficacy of the joint deep neural network in both the tasks of anemia classification and Hb levels (g/dL) estimation.