Some of the operator product expansions (OPEs) between the lowest 16 higher spin currents of spins (1, 3/2, 3/2, 3/2, 3/2, 2, 2, 2, 2, 2, 2, 5/2, 5/2, 5/2, 5/2, 3) in an extension of the large N=4 linear superconformal algebra were constructed in the N=4 superconformal coset SU(5)/SU(3) theory previously. In this paper, by rewriting the above OPEs in the N=4 superspace developed by Schoutens (and other groups), the remaining undetermined OPEs where the corresponding singular terms possess the composite fields with spins s =7/2, 4, 9/2, 5 are completely determined. Furthermore, by introducing the arbitrary coefficients in front of the composite fields in the right hand sides of the above complete 136 OPEs, reexpressing them in the N=2 superspace and using the N=2 OPEs mathematica package by Krivonos and Thielemans, the complete structures of the above OPEs with fixed coefficient functions are obtained with the help of various Jacobi identities. Then one obtains ten N=2 super OPEs between the four N=2 higher spin currents denoted by (1, 3/2, 3/2, 2), (3/2, 2, 2, 5/2), (3/2, 2, 2, 5/2) and (2, 5/2, 5/2, 3) (corresponding 136 OPEs in the component approach) in the N=4 superconformal coset SU(N+2)/SU(N) theory. Finally, one describes them as one single N=4 super OPE between the above sixteen higher spin currents in the N=4 superspace. The fusion rule for this OPE contains the next 16 higher spin currents of spins of (2, 5/2, 5/2, 5/2, 5/2, 3, 3, 3, 3, 3, 3, 7/2, 7/2, 7/2, 7/2, 4) in addition to the quadratic N=4 lowest higher spin multiplet and the large N=4 linear superconformal family of the identity operator. The various structure constants (fixed coefficient functions) appearing in the right hand side of this OPE depend on N and the level k of the bosonic spin-1 affine Kac-Moody current.