Point interactions for the second derivative operator in one dimension are studied. Every operator from this family is described by the boundary conditions which include a 2x2 real matrix with the unit determinant and a phase. The role of the phase parameter leading to unitarily equivalent operators is discussed in the present paper. In particular, it is shown that the phase parameter is not redundant (contrary to previous studies) if nonstationary problems are concerned. It is proven that the phase parameter can be interpreted as the amplitude of a singular gauge field. Considering the few-body problem we extend the range of parameters for which the exact solution can be found using the Bethe Ansatz.