The $n$-dimensional pair of pants is defined to be the complement of $n + 2$ generic hyperplanes in $\mathbb{CP}n$. We construct an immersed Lagrangian sphere in the pair of pants and compute its endomorphism $A_\infty$ algebra in the Fukaya category. On the level of cohomology, it is an exterior algebra with $n+2$ generators. It is not formal, and we compute certain higher products in order to determine it up to quasi-isomorphism. This allows us to give some evidence for the Homological Mirror Symmetry conjecture: the pair of pants is conjectured to be mirror to the Landau-Ginzburg model $(\mathbb{C}^{n+2},W)$, where $W = z_1...z_{n+2}$. We show that the endomorphism $A_\infty$ algebra of our Lagrangian is quasi-isomorphic to the endomorphism dg algebra of the structure sheaf of the origin in the mirror. This implies similar results for finite covers of the pair of pants, in particular for certain affine Fermat hypersurfaces.