Chunk segmentation of Chinese sentences using a Combined Statistical and Rule-based Approach (CSRA)

R. Wang, X. Wang, Z. Chen, Zheru Chi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Deep parsing of Chinese sentences is a very challenging task due to their complexity such as ambiguous word boundaries and meanings. An alternative mode of Chinese language processing is to perform shallow parsing of Chinese sentences in which chunk segmentation plays an important role. In this paper, we present a chunk segmentation algorithm using a combined statistical and rule-based approach (CSRA). The decision rules for refining chunk segmentation are generated from incorrectly segmented chunks from a statistical model which is built on a training corpus. Experimental results show that the CSRA works well and produces satisfactory chunk segmentation results for subsequent processes such as chunk tagging and chunk collocation extraction.
Original languageEnglish
Pages (from-to)197-218
Number of pages22
JournalInternational journal of computer processing of languages
Volume20
Issue number2
DOIs
Publication statusPublished - 2007

Keywords

  • Chinese language processing
  • Chinese chunk segmentation
  • Machine translation
  • Language modeling
  • Decision rules

Cite this