The recognition of human activities of daily living has gained increasing attention in recent years due to its potential to promote autonomy for elderly people in their own homes and its usage for security surveillance in scenarios such as supermarkets, banks, etc. Infrared thermal array has been proposed as a non-invasive device for human activity detection, which has the advantages of low-cost, low-power, and high-performance capabilities. However, most of the ambient-based sensor research has not considered animal pets, whose bodies have similar temperature to human body, that may live with the elderly people in a single inhabitant environment. This has led to a gap in the usability and deployability of such systems on a broader range. This paper proposes a filtering method for removing thermal noises, including those radiated by pets, from the acquired heat-map. Therefore, only the presence of a human in the thermal scene is considered for further activity recognition. This paper shows the possibility of using an entropy point estimate for a one-dimensional heat-map histogram as a distinctive attribute to distinguish between heat-maps of the human and animal pet with 100% achieved accuracy.