We present Atacama Large Millimeter/submillimeter Array (ALMA) Band 7 observations of a remarkably bright galaxy candidate at (M UV = −21.6), S5-z17-1, identified in James Webb Space Telescope (JWST) Early Release Observation data of Stephen’s Quintet. We do not detect the dust continuum at 866 μm, ruling out the possibility that S5-z17-1 is a low-z dusty starburst with a star formation rate of ≳30 M ⊙ yr−1. We detect a 5.1σ line feature at 338.726 ± 0.007 GHz exactly coinciding with the JWST source position, with a 2% likelihood of the signal being spurious. The most likely line identification would be [O iii]52 μm at z = 16.01 or [C ii]158 μm at z = 4.61, whose line luminosities do not violate the nondetection of the dust continuum in both cases. Together with three other z ≳ 11–13 candidate galaxies recently observed with ALMA, we conduct a joint ALMA and JWST spectral energy distribution (SED) analysis and find that the high-z solution at z ∼ 11–17 is favored in every candidate as a very blue (UV continuum slope of ≃−2.3) and luminous (M UV ≃ [ − 24:−21]) system. Still, we find in several candidates that reasonable SED fits (Δχ 2 ≲ 4) are reproduced by type II quasar and/or quiescent galaxy templates with strong emission lines at z ∼ 3–5, where such populations predicted from their luminosity functions and EW([O iii]+Hβ) distributions are abundant in survey volumes used for the identification of the z ∼ 11–17 candidates. While these recent ALMA observation results have strengthened the likelihood of the high-z solutions, lower-z possibilities are not completely ruled out in several of the z ∼ 11–17 candidates, indicating the need to consider the relative surface densities of the lower-z contaminants in the ultra-high-z galaxy search.