Understandings of violence, and especially sexual violence against children, must be situated within the local context. The 2009 Violence against Children Survey in Zanzibar indicated that 6% of girls and 9% of boys reported having experienced sexual violence before the age of 18 years. This paper reports on an in-depth qualitative study conducted in Zanzibar to provide further insights to these findings by examining the circumstances for sexual and other violence against children in Zanzibar. Twenty-four in-depth interviews with young people and 18 focus-group discussions with young people and adults were conducted in rural and urban Zanzibar. A further 8 interviews were conducted with parents and key stakeholders in government and NGO offices that provide services for children. The findings revealed that religious and cultural practices, which form the foundation of Swahili culture in Zanzibar, provide a moral frame for childhood development, but structural factors make children vulnerable to sexual violence. Both boys and girls are vulnerable to sexual violence in the home, neighbourhood, at school and, in particular, at madrasa or Qur’anic schools. As religion and culture are strong influences on childhood, preventing sexual violence at madrasa schools would strengthen the positive aspects of religious teachings for ensuring a safe childhood.