We report on the magnetic properties of single crystals of Ho- and Dy-based ${({\mathrm{Mo}}_{2/3}{R}_{1/3})}_{2}\mathrm{AlC} i\text{\ensuremath{-}}\mathrm{MAX}$ phases. In these nanolamellar compounds, where planes of $R$ and Mo arranged in a skewed triangular lattice are separated by planes of Al and C, geometrical frustration and magnetic exchange interactions lead to complex magnetic properties. Temperature-dependent bulk magnetization, specific heat, and resistivity measurements reveal two magnetic phase transitions in Dy $i\text{\ensuremath{-}}\mathrm{MAX}$ (15 and 12 K) and only one in Ho $i\text{\ensuremath{-}}\mathrm{MAX}$ (8.5 K). Strong magnetic anisotropy and metamagnetic transitions with a step at $\frac{1}{3}$ of saturation moment along the crystal $a$ axis are observed in field-dependent bulk magnetization curves. X-ray magnetic circular dichroism measurements unveil induced moments on Mo and Al, and a quantitative estimation of the orbital and spin moments of Mo based on magneto-optical sum rules suggests an unusual interaction between the $R 4f$ and the Mo $4d$ magnetic moments. Magnetic structures are derived from neutron diffraction measurements, revealing a zero-field incommensurate amplitude modulated order in both compounds, followed by an antiferromagnetic equal-moments structure at lower temperature for Dy $i\text{\ensuremath{-}}\mathrm{MAX}$. The bulk magnetization $\frac{1}{3}$ step is found to be linked to the flip of one $R$ moment out of three within the planes. Detailed phase diagrams for Ho and Dy $i\text{\ensuremath{-}}\mathrm{MAX}$ are derived from these measurements.