We study a large $N$ tensor model with $O(N)^3$ symmetry containing two flavors of Majorana fermions, $\psi_1^{abc}$ and $\psi_2^{abc}$. We also study its random counterpart consisting of two coupled Sachdev-Ye-Kitaev models, each one containing $N_{\rm SYK}$ Majorana fermions. In these models we assume tetrahedral quartic Hamiltonians which depend on a real coupling parameter $\alpha$. We find a duality relation between two Hamiltonians with different values of $\alpha$, which allows us to restrict the model to the range of $-1\leq \alpha\leq 1/3$. The scaling dimension of the fermion number operator $Q=i\psi_1^{abc} \psi_2^{abc}$ is complex and of the form $1/2 +i f(\alpha)$ in the range $-1\leq \alpha<0$, indicating an instability of the conformal phase. Using Schwinger-Dyson equations to solve for the Green functions, we show that in the true low-temperature phase this operator acquires an expectation value. This demonstrates the breaking of an anti-unitary particle-hole symmetry and other discrete symmetries. We also calculate spectra of the coupled SYK models for values of $N_{\rm SYK}$ where exact diagonalizations are possible. For negative $\alpha$ we find a gap separating the two lowest energy states from the rest of the spectrum; this leads to exponential decay of the zero-temperature correlation functions. For $N_{\rm SYK}$ divisible by $4$, the two lowest states have a small splitting. They become degenerate in the large $N_{\rm SYK}$ limit, as expected from the spontaneous breaking of a $\mathbb{Z}_2$ symmetry.