Millimeter wave (mmWave) communications holds a promise to offer an unprecedented capacity boost for 5G cellular networks. Due to the small wavelength of mmWave signals, mmWave multiple-input-multiple-output (MIMO) systems can leverage large-scale antennas to combat the path loss and rain attenuation via precoding. Different from conventional MIMO systems, mmWave MIMO cannot realize precoding entirely at baseband using digital precoders, as a result of potentially high power consumed by signal mixers and analog-to-digital converters (ADCs). As a cost- effective alternative, a hybrid precoding transceiver architecture for mmWave MIMO systems has received considerable attention. However, the optimal design of such hybrid precoding has not been fully understood. In this paper, an alternating minimization algorithm based on manifold optimization is proposed to design the hybrid precoder, thereby making it comparable in performance to the digital precoder. Numerical results show that our proposed algorithm can significantly outperform existing ones in terms of spectral efficiency and, more importantly, it can achieve the optimal performance in certain cases. Finally, the alternating minimization approach is shown to be generally applicable to precoding design with different hybrid structures, and the corresponding comparison will show interesting design insights for hybrid precoding.