Word boundary decision with CRF for Chinese word segmentation

Shoushan Li; Chu-ren Huang

Word boundary decision with CRF for Chinese word segmentation

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

3 Citations (Scopus)

Abstract

Chinese word segmentation systems necessarily perform both accurately and quickly for real applications. In this paper, we study on word boundary decision (WBD) approach for Chinese word segmentation and implement it as a 2-tag character tagging with conditional random filed (CRF). With a help of tag transition features, WBD with CRF segmentation approach can achieve comparative performances compared to 4-tag character tagging approach (represents the state-of-the-art segmentation approach). But it requires only about half training time and memory space as much as 4-tag character tagging approach. These results encourage that WBD segmentation approach is a good choice for real Chinese word segmentation systems.

Original language	English
Title of host publication	PACLIC 23 - Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation
Pages	726-732
Number of pages	7
Volume	2
Publication status	Published - 1 Dec 2009
Event	23rd Pacific Asia Conference on Language, Information and Computation, PACLIC 23 - Hong Kong, Hong Kong Duration: 3 Dec 2009 → 5 Dec 2009

Conference

Conference	23rd Pacific Asia Conference on Language, Information and Computation, PACLIC 23
Country/Territory	Hong Kong
City	Hong Kong
Period	3/12/09 → 5/12/09

Keywords

Chinese word segmentation
Conditional random field
Word boundary decision

ASJC Scopus subject areas

Language and Linguistics
Computer Science (miscellaneous)

Cite this

@inproceedings{8323ce2c656648b981d8434dc78f26a3,

title = "Word boundary decision with CRF for Chinese word segmentation",

abstract = "Chinese word segmentation systems necessarily perform both accurately and quickly for real applications. In this paper, we study on word boundary decision (WBD) approach for Chinese word segmentation and implement it as a 2-tag character tagging with conditional random filed (CRF). With a help of tag transition features, WBD with CRF segmentation approach can achieve comparative performances compared to 4-tag character tagging approach (represents the state-of-the-art segmentation approach). But it requires only about half training time and memory space as much as 4-tag character tagging approach. These results encourage that WBD segmentation approach is a good choice for real Chinese word segmentation systems.",

keywords = "Chinese word segmentation, Conditional random field, Word boundary decision",

author = "Shoushan Li and Chu-ren Huang",

year = "2009",

month = dec,

day = "1",

language = "English",

isbn = "9789624423198",

volume = "2",

pages = "726--732",

booktitle = "PACLIC 23 - Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation",

note = "23rd Pacific Asia Conference on Language, Information and Computation, PACLIC 23 ; Conference date: 03-12-2009 Through 05-12-2009",

}

Word boundary decision with CRF for Chinese word segmentation. / Li, Shoushan; Huang, Chu-ren.
PACLIC 23 - Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation. Vol. 2 2009. p. 726-732.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Word boundary decision with CRF for Chinese word segmentation

AU - Li, Shoushan

AU - Huang, Chu-ren

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Chinese word segmentation systems necessarily perform both accurately and quickly for real applications. In this paper, we study on word boundary decision (WBD) approach for Chinese word segmentation and implement it as a 2-tag character tagging with conditional random filed (CRF). With a help of tag transition features, WBD with CRF segmentation approach can achieve comparative performances compared to 4-tag character tagging approach (represents the state-of-the-art segmentation approach). But it requires only about half training time and memory space as much as 4-tag character tagging approach. These results encourage that WBD segmentation approach is a good choice for real Chinese word segmentation systems.

AB - Chinese word segmentation systems necessarily perform both accurately and quickly for real applications. In this paper, we study on word boundary decision (WBD) approach for Chinese word segmentation and implement it as a 2-tag character tagging with conditional random filed (CRF). With a help of tag transition features, WBD with CRF segmentation approach can achieve comparative performances compared to 4-tag character tagging approach (represents the state-of-the-art segmentation approach). But it requires only about half training time and memory space as much as 4-tag character tagging approach. These results encourage that WBD segmentation approach is a good choice for real Chinese word segmentation systems.

KW - Chinese word segmentation

KW - Conditional random field

KW - Word boundary decision

UR - http://www.scopus.com/inward/record.url?scp=84863880020&partnerID=8YFLogxK

M3 - Conference article published in proceeding or book

SN - 9789624423198

VL - 2

SP - 726

EP - 732

BT - PACLIC 23 - Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation

T2 - 23rd Pacific Asia Conference on Language, Information and Computation, PACLIC 23

Y2 - 3 December 2009 through 5 December 2009

ER -

Word boundary decision with CRF for Chinese word segmentation

Abstract

Conference

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this