Scene modelling is of great importance for robots in unknown environments. Existing Visual Simultaneous Localization and Mapping (Visual SLAM) approaches are able to build impressive scene models using RGB-D cameras in static scenes. In dynamic scenes, however, moving objects can be recorded as spurious objects, which contaminates the resulting scene models. In order to build clear scene models, we propose a novel moving-object removal approach for scene modelling algorithms in this paper. Our approach does not rely on prior knowledge, such as appearance features or initial segmentation. In addition, the proposed approach does not require an initialization process, which is different from most background subtraction algorithms. The experimental results demonstrate that our approach is able to effectively remove moving objects and assist scene modelling algorithms to build clear models in dynamic scenes.