Young Black men who have sex with men (MSM) and transgender people experience disparities in HIV incidence and HIV outcomes. To effectively engage these communities in HIV-related programming, we developed a recreation-based community health space. We sought to examine the challenges and successes in implementing this program. Qualitative data were collected from federal progress reports and by a process evaluator who recorded interviews with staff, stakeholders, and participants. These data were coded for themes related to barriers and successes. We consolidated themes into four key domains: community engagement, service provision, stigma, and violence. Each of these domains was determined to significantly affect programmatic success during the implementation period. Young Black MSM and transgender people experience stigmas that pose challenges to effective engagement in HIV-related programming. These lessons learned offer strategies for community engagement and for addressing violence and stigma to maximize programmatic effectiveness.