ATPTrack: Visual tracking with alternating token pruning of dynamic templates and search region

Constantly varying appearance of targets brings tremendous challenges for visual object tracking, especially in long-term tracking and background interference scenarios. Current leading trackers attempt to introduce multi-level fixed and dynamic templates to encode changing target information and abundant spatiotemporal contexts. These methods achieve astonishing performance improvements. However, dynamic templates are obtained from intermediate frames with high tracking confidence scores. These frames are not manually annotated. Therefore, the accuracy of dynamic templates relies entirely on tracking results. Dynamic templates may contain a large amount of uninformative and irrelevant background noise due to imprecise tracking. Additionally, multiple templates result in increased computational complexity as well. In order to tackle the above problems, a novel tracker dubbed ATPTrack is proposed for efficient end-to-end tracking. ATPTrack combines the initial target information of fixed templates and updated target representations of dynamic templates. Particularly, ATPTrack develops an alternating token trimming method that prunes dynamic templates and search region progressively. After token simplification, target-related information is highlighted in both dynamic templates and search region, which makes it more computationally efficient. Furthermore, a novel similarity ranking module is incorporated into the dynamic template pruning (DTP) block and the search region pruning (SRP) block for selecting the most discriminative target tokens. The DTP and SRP blocks are responsible for efficient trimming of dynamic templates and the search region, separately. The proposed ATPTrack achieves competitive performance on multiple tracking benchmarks. Compared to merely trimming the search region, ATPTrack further reduces the multiply-accumulate operations (MACs) by 11.5 % with negligible performance drop of 0.3 % by alternately pruning dynamic templates and search region.