Hybrid precoding is a cost-effective approach to support directional transmissions for millimeter wave (mmWave) communications. While existing works on hybrid precoding mainly focus on single-user single-carrier transmission, in practice multicarrier transmission is needed to combat the much increased bandwidth, and multiuser MIMO can provide additional spatial multiplexing gains. In this paper, we propose a new hybrid precoding structure for multiuser OFDM mmWave systems, which greatly simplifies the hybrid precoder design and is able to approach the performance of the fully digital precoder. In particular, two groups of phase shifters are combined to map the signals from radio frequency (RF) chains to antennas. Then an effective hybrid precoding algorithm based on alternating minimization (AltMin) is proposed, which will alternately optimize the digital and analog precoders. A major algorithmic innovation is a LASSO formulation for the analog precoder, which yields computationally efficient algorithms. Simulation results will show the performance gain of the proposed algorithm. Moreover, it will reveal that canceling the interuser interference is critical in multiuser OFDM hybrid precoding systems.