Indexed spatio-temporal appearance models for query-driven video action recognition

Video action and event recognition is an important problem in video analysis research with many important applications, such as surveillance and video search. In this work, we deal with the appearance complexity in video action recognition by applying an indexing structure and partition in appearance space. The task requires spatio-temporal appearance modeling that can capture the discriminative information among different action classes. Traditional approaches are based on a global appearance model, which is not robust to local variations in background. In this work, we develop a query driven dynamic appearance modeling method and use a localized subspace to obtain a distance metric for appearance discrimination. Multiple localized models are constructed and utilized to measure the similarity between the trajectories and the sub-space metric is adaptive during the learning process. The processing is implemented based on an indexing scheme, which is very fast in computation. Simulation results demonstrate the effectiveness of the solution.