Hybrid precoding has been recently proposed as a cost- effective transceiver solution for millimeter wave (mm- wave) systems. The analog component in such precoders, which is composed of a phase shifter network, is the key differentiating element in contrast to conventional fully digital precoders. While a large number of phase shifters with unquantized phases are commonly assumed in existing works, in practice the phase shifters should be discretized with a coarse quantization, and their number should be reduced to a minimum due to cost and power consideration. In this paper, we propose a new hybrid precoder implementation using a small number of phase shifters with quantized and fixed phases, i.e., a fixed phase shifter (FPS) implementation, which significantly reduces the cost and hardware complexity. In addition, a dynamic switch network is proposed to enhance the spectral efficiency. Based on the proposed FPS implementation, an effective alternating minimization (AltMin) algorithm is developed with closed-form solutions in each iteration. Simulation results show that the proposed algorithm with the FPS implementation outperforms existing ones. More importantly, it needs much fewer phase shifters than existing hybrid precoder proposals, e.g., ~10 fixed phase shifters are sufficient for practically relevant system settings.