This paper is concerned with the stability of a planar traveling wave in a cylindrical domain. The equation describes activator-inhibitor systems in chemistry or biology. The wave has a thin transition layer and is constructed by singular perturbation methods. Let $\varepsilon$ be the width of the layer. We show that, if the cross section of the domain is narrow enough, the traveling wave is asymptotically stable, while it is unstable if the cross section is wide enough by studying the linearized eigenvalue problem. For the latter case, we study the wavelength associated with an eigenvalue with the largest real part, which is called the fastest growing wavelength. We prove that this wavelength is $O(\varepsilon^{1/3})$ as $\varepsilon$ goes to zero mathematically rigorously. This fact shows that, if unstable planar waves are perturbed randomly, this fastest growing wavelength is selectively amplified with as time goes on. For this analysis, we use a new uniform convergence theorem for some inverse operator and carry out the Lyapunov-Schmidt reduction.