In this paper, we consider a simple and low-complexity discrete first-order method called the Generalized Power Method (GPM) for large-scale MIMO detection. The GPM is essentially a projected gradient method and exploits the fact that the projection onto the discrete MPSK or QAM constellation is efficiently computable. As our main contribution, we first show that under certain conditions on the channel and additive noise, the GPM will converge to the true symbol vector in a finite number of iterations. We then show that the aforementioned conditions will be satisfied with high probability under standard probabilistic models of the channel and noise. Besides enjoying strong theoretical guarantees, the proposed method is shown in our simulations to be competitive with existing methods in terms of both detection performance and numerical efficiency. We believe that our techniques will find further applications in the development of high-performance detection methods for massive MIMO.