Exploring spatial correlation for visual object retrieval

Miaojing Shi, Xinghai Sun, Dacheng Tao, Chao Xu, George Baciu, Hong Liu

Research output: Journal article publicationJournal articleAcademic researchpeer-review

4 Citations (Scopus)


Bag-of-visual-words (BOVW)-based image representation has received intense attention in recent years and has improved content-based image retrieval (CBIR) significantly. BOVW does not consider the spatial correlation between visual words in natural images and thus biases the generated visual words toward noise when the corresponding visual features are not stable. This article outlines the construction of a visual word co-occurrence matrix by exploring visual word co-occurrence extracted from small affine-invariant regions in a large collection of natural images. Based on this co-occurrence matrix, we first present a novel high-order predictor to accelerate the generation of spatially correlated visual words and a penalty tree (PTree) to continue generating the words after the prediction. Subsequently, we propose two methods of co-occurrence weighting similarity measure for image ranking: Co-Cosine and Co-TFIDF. These two new schemes downweight the contributions of the words that are less discriminative because of frequent co-occurrences with other words.We conduct experiments on Oxford and Paris Building datasets, in which the ImageNet dataset is used to implement a large-scale evaluation. Cross-dataset evaluations between the Oxford and Paris datasets and Oxford and Holidays datasets are also provided. Thorough experimental results suggest that our method outperforms the state of the art without adding much additional cost to the BOVW model.
Original languageEnglish
Article number24
JournalACM Transactions on Intelligent Systems and Technology
Issue number2
Publication statusPublished - 31 Mar 2015


  • BOVW
  • Co-Cosine
  • Co-TFIDF
  • High-order predictor
  • Penalty tree
  • Spatial correlation

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Artificial Intelligence


Dive into the research topics of 'Exploring spatial correlation for visual object retrieval'. Together they form a unique fingerprint.

Cite this