Huntington’s disease (HD) is a genetically caused neurodegenerative disorder characterized by heterogeneous motor, psychiatric, and cognitive symptoms. Although motor symptoms may be the most prominent presentation, cognitive symptoms such as memory deficits and executive dysfunction typically co-occur. We used functional magnetic resonance imaging (fMRI) and task fMRI-based dynamic causal modeling (DCM) to evaluate HD-related changes in the neural network underlying working memory (WM). Sixty-four pre-symptomatic HD mutation carriers (preHD), 20 patients with early manifest HD symptoms (earlyHD), and 83 healthy control subjects performed an n-back fMRI task with two levels of WM load. Effective connectivity was assessed in five predefined regions of interest, comprising bilateral inferior parietal cortex, left anterior cingulate cortex, and bilateral dorsolateral prefrontal cortex. HD mutation carriers performed less accurately and more slowly at high WM load compared with the control group. While between-group comparisons of brain activation did not reveal differential recruitment of the cortical WM network in mutation carriers, comparisons of brain connectivity as identified with DCM revealed a number of group differences across the whole WM network. Most strikingly, we observed decreasing connectivity from several regions toward right dorsolateral prefrontal cortex (rDLPFC) in preHD and even more so in earlyHD. The deterioration in rDLPFC connectivity complements results from previous studies and might mirror beginning cortical neural decline at premanifest and early manifest stages of HD. We were able to characterize effective connectivity in a WM network of HD mutation carriers yielding further insight into patterns of cognitive decline and accompanying neural deterioration.