Learningword embeddings via context grouping

Yun Ma, Qing Li, Zhenguo Yang, Wenyin Liu, Antoni B. Chan

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Recently, neural-network based word embedding models have been shown to produce high-quality distributional representations capturing both semantic and syntactic information. In this paper, we propose a grouping-based context predictive model by considering the interactions of contextwords, which generalizes the widely used CBOWmodel and Skip-Gram model. In particular, the words within a context window are split into several groups with a grouping function, where words in the same group are combined while different groups are treated as independent. To determine the grouping function, we propose a relatedness hypothesis stating the relationship among context words and propose several context grouping methods. Experimental results demonstrate better representations can be learned with suitable context groups.

Original languageEnglish
Title of host publicationProceedings of the ACM Turing 50th Celebration Conference - China, ACM TUR-C 2017
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450348737
DOIs
Publication statusPublished - 12 May 2017
Externally publishedYes
Event50th ACM Turing Conference - China, ACM TUR-C 2017 - Shanghai, China
Duration: 12 May 201714 May 2017

Publication series

NameACM International Conference Proceeding Series
VolumePart F127754

Conference

Conference50th ACM Turing Conference - China, ACM TUR-C 2017
Country/TerritoryChina
CityShanghai
Period12/05/1714/05/17

Keywords

  • Context grouping
  • Non-parametric clustering
  • Word embeddings

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this