The method of window exchange umbrella sampling molecular dynamics (WEUSMD) with a pre-optimized parameter set was recently used to obtain the most probable conformations and the energetics of transmembrane (TM) helix assembly of a generic TM sequence. When applied to glycophorin A TM domain (GpA-TM) using the window potentials along the helix-helix distance, however, tight interfacial packing of GpA-TM resulted in insufficient conformational sampling at short helix-helix separation. To address this sampling issue, we approached in two directions: (i) Extension of WEUSMD into two dimensions with the restraint potentials along the rHH and crossing angle (Ω) to bypass barriers for hidden variables; (ii) Design of another scheme for the window spacing rather than a uniform one to control the average acceptance probability (Pa) between neighboring windows systematically. The two-dimensional WEUSMD results demonstrate that the incomplete sampling in the one-dimensional WEUSMD arises from high barriers along the crossing angle between the GpA-TM helices. In principle, the multi-dimensional WEUSMD is suitable for modeling TM helix assembly. However, for three or higher dimensions, this approach becomes prohibitively intensive. To avoid this issue, one may consider the latter approach, the variable window spacing for WEUSMD, where the (highest) restraint force constant for the window at the shortest rHH geometrically decreases with rHH and the first passage time optimized parameter set is estimated by using an analytic approximation of Pa in general case (different window force constants for the exchange pairs). To demonstrate the improved sampling power of WEUSMD that in turn provides trustworthy potential of mean force, we applied WEUSMD with variable window spacing for the assembly of GpA-TM, and indeed obtained the improved sampling power compared to that with uniform window spacing.