A comparative study of the effect of word segmentation on chinese terminology extraction

Luning Ji; Qin Lu; Wenjie Li; YiRong Chen

A comparative study of the effect of word segmentation on chinese terminology extraction

Luning Ji, Qin Lu, Wenjie Li, YiRong Chen

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

Abstract

Automatic term extraction is the first step towards automatic or semi-automatic update of existing domain knowledge base. Most of the researches applied word segmentation as a preprocessing step to Chinese term extraction. However, segmentation ambiguity is unavoidable, especially in identifying unknown words for Chinese. In this paper, we discuss the effect and limitations of segmentation to Chinese terminology extraction. Detailed study shows that propagated errors caused by word segmentation have great impact on the result of terminology extraction. Based on our analysis and experiments, it is proven that character-based terminology extraction yields much better result than that using segmentation as a preprocessing step.

Original language	English
Title of host publication	PACLIC 20 - Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation
Pages	101-109
Number of pages	9
Publication status	Published - 1 Dec 2006
Event	20th Pacific Asia Conference on Language, Information and Computation, PACLIC 20 - Wuhan, China Duration: 1 Nov 2006 → 3 Nov 2006

Conference

Conference	20th Pacific Asia Conference on Language, Information and Computation, PACLIC 20
Country/Territory	China
City	Wuhan
Period	1/11/06 → 3/11/06

Keywords

Chinese corpus
Terminology extraction
Word segmentation

ASJC Scopus subject areas

Language and Linguistics
Computer Science (miscellaneous)

Cite this

@inproceedings{2a3a7176e46c40049de68b4a2d39762f,

title = "A comparative study of the effect of word segmentation on chinese terminology extraction",

abstract = "Automatic term extraction is the first step towards automatic or semi-automatic update of existing domain knowledge base. Most of the researches applied word segmentation as a preprocessing step to Chinese term extraction. However, segmentation ambiguity is unavoidable, especially in identifying unknown words for Chinese. In this paper, we discuss the effect and limitations of segmentation to Chinese terminology extraction. Detailed study shows that propagated errors caused by word segmentation have great impact on the result of terminology extraction. Based on our analysis and experiments, it is proven that character-based terminology extraction yields much better result than that using segmentation as a preprocessing step.",

keywords = "Chinese corpus, Terminology extraction, Word segmentation",

author = "Luning Ji and Qin Lu and Wenjie Li and YiRong Chen",

year = "2006",

month = dec,

day = "1",

language = "English",

pages = "101--109",

booktitle = "PACLIC 20 - Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation",

note = "20th Pacific Asia Conference on Language, Information and Computation, PACLIC 20 ; Conference date: 01-11-2006 Through 03-11-2006",

}

A comparative study of the effect of word segmentation on chinese terminology extraction. / Ji, Luning; Lu, Qin; Li, Wenjie et al.

PACLIC 20 - Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation. 2006. p. 101-109.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - A comparative study of the effect of word segmentation on chinese terminology extraction

AU - Ji, Luning

AU - Lu, Qin

AU - Li, Wenjie

AU - Chen, YiRong

PY - 2006/12/1

Y1 - 2006/12/1

N2 - Automatic term extraction is the first step towards automatic or semi-automatic update of existing domain knowledge base. Most of the researches applied word segmentation as a preprocessing step to Chinese term extraction. However, segmentation ambiguity is unavoidable, especially in identifying unknown words for Chinese. In this paper, we discuss the effect and limitations of segmentation to Chinese terminology extraction. Detailed study shows that propagated errors caused by word segmentation have great impact on the result of terminology extraction. Based on our analysis and experiments, it is proven that character-based terminology extraction yields much better result than that using segmentation as a preprocessing step.

AB - Automatic term extraction is the first step towards automatic or semi-automatic update of existing domain knowledge base. Most of the researches applied word segmentation as a preprocessing step to Chinese term extraction. However, segmentation ambiguity is unavoidable, especially in identifying unknown words for Chinese. In this paper, we discuss the effect and limitations of segmentation to Chinese terminology extraction. Detailed study shows that propagated errors caused by word segmentation have great impact on the result of terminology extraction. Based on our analysis and experiments, it is proven that character-based terminology extraction yields much better result than that using segmentation as a preprocessing step.

KW - Chinese corpus

KW - Terminology extraction

KW - Word segmentation

UR - http://www.scopus.com/inward/record.url?scp=84863643074&partnerID=8YFLogxK

M3 - Conference article published in proceeding or book

SP - 101

EP - 109

BT - PACLIC 20 - Proceedings of the 20th Pacific Asia Conference on Language, Information and Computation

T2 - 20th Pacific Asia Conference on Language, Information and Computation, PACLIC 20

Y2 - 1 November 2006 through 3 November 2006

ER -

A comparative study of the effect of word segmentation on chinese terminology extraction

Abstract

Conference

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this