Automatic plant leaf recognition has been a hot research spot in the recent years, where encouraging improvements have been achieved in both recognition accuracy and speed. However, existing algorithms usually only extracted leaf features (such as shape or texture) or merely adopt traditional neural network algorithm to recognize leaf, which still showed limitation in recognition accuracy and speed especially when facing a large leaf database. In this paper, we present a novel method for leaf recognition by combining feature extraction and machine learning. To break the weakness exposed in the traditional algorithms, we applied binary Gabor pattern (BGP) and extreme learning machine (ELM) to recognize leaves. To accelerate the leaf recognition, we also extract BGP features from leaf images with an offline manner. Different from the traditional neural network like BP and SVM, our method based on the ELM only requires setting one parameter, and without additional fine-tuning during the leaf recognition. Our method is evaluated on several different databases with different scales. Comparisons with state-of-the-art methods were also conducted to evaluate the combination of BGP and ELM. Visual and statistical results have demonstrated its effectiveness.