An automatic Chinese collocation extraction algorithm based on lexical statistics

Ruifeng Xu; Qin Lu; Yin Li

doi:10.1109/NLPKE.2003.1275923

An automatic Chinese collocation extraction algorithm based on lexical statistics

Ruifeng Xu, Qin Lu, Yin Li

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

12 Citations (Scopus)

Abstract

This paper presents an automatic Chinese collocation extraction system using lexical statistics and syntactical knowledge. This system extracts collocations from manually segmented and tagged Chinese news corpus in three stages. First, the BI-directional BI-Gram statistical measures, including BI-directional strength and spread, and x2 test value, are employed to extract candidate two-word pairs. These candidate word pairs are then used to extract high frequency multi-word collocations from their context. In the third stage, precision is further improved by using syntactical knowledge of collocation patterns between content words to eliminate pseudo collocations. In the preliminary experiment on 30 selected headwords, this three-stage system achieves a 73% precision rate, a substantial improvement on the 61% achieved using an algorithm we developed earlier based on an improved version of the Smdja's 53% accurate Xtract system.

Original language	English
Title of host publication	NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings
Publisher	IEEE
Pages	321-326
Number of pages	6
ISBN (Electronic)	0780379020, 9780780379022
DOIs	https://doi.org/10.1109/NLPKE.2003.1275923
Publication status	Published - 1 Jan 2003
Event	International Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003 - Beijing Media Center, Beijing, China Duration: 26 Oct 2003 → 29 Oct 2003

Conference

Conference	International Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003
Country/Territory	China
City	Beijing
Period	26/10/03 → 29/10/03

Keywords

Chinese collocation
Information extraction and statistical models

ASJC Scopus subject areas

Artificial Intelligence
Computational Theory and Mathematics
Software

More information

10.1109/NLPKE.2003.1275923

Cite this

@inproceedings{739655baf2d9440b9625c6c2a7d12df1,

title = "An automatic Chinese collocation extraction algorithm based on lexical statistics",

abstract = "This paper presents an automatic Chinese collocation extraction system using lexical statistics and syntactical knowledge. This system extracts collocations from manually segmented and tagged Chinese news corpus in three stages. First, the BI-directional BI-Gram statistical measures, including BI-directional strength and spread, and x2 test value, are employed to extract candidate two-word pairs. These candidate word pairs are then used to extract high frequency multi-word collocations from their context. In the third stage, precision is further improved by using syntactical knowledge of collocation patterns between content words to eliminate pseudo collocations. In the preliminary experiment on 30 selected headwords, this three-stage system achieves a 73% precision rate, a substantial improvement on the 61% achieved using an algorithm we developed earlier based on an improved version of the Smdja's 53% accurate Xtract system.",

keywords = "Chinese collocation, Information extraction and statistical models",

author = "Ruifeng Xu and Qin Lu and Yin Li",

year = "2003",

month = jan,

day = "1",

doi = "10.1109/NLPKE.2003.1275923",

language = "English",

pages = "321--326",

booktitle = "NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings",

publisher = "IEEE",

address = "United States",

note = "International Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003 ; Conference date: 26-10-2003 Through 29-10-2003",

}

Xu, R, Lu, Q & Li, Y 2003, An automatic Chinese collocation extraction algorithm based on lexical statistics. in NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings., 1275923, IEEE, pp. 321-326, International Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003, Beijing, China, 26/10/03. https://doi.org/10.1109/NLPKE.2003.1275923

An automatic Chinese collocation extraction algorithm based on lexical statistics. / Xu, Ruifeng; Lu, Qin; Li, Yin.
NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings. IEEE, 2003. p. 321-326 1275923.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - An automatic Chinese collocation extraction algorithm based on lexical statistics

AU - Xu, Ruifeng

AU - Lu, Qin

AU - Li, Yin

PY - 2003/1/1

Y1 - 2003/1/1

N2 - This paper presents an automatic Chinese collocation extraction system using lexical statistics and syntactical knowledge. This system extracts collocations from manually segmented and tagged Chinese news corpus in three stages. First, the BI-directional BI-Gram statistical measures, including BI-directional strength and spread, and x2 test value, are employed to extract candidate two-word pairs. These candidate word pairs are then used to extract high frequency multi-word collocations from their context. In the third stage, precision is further improved by using syntactical knowledge of collocation patterns between content words to eliminate pseudo collocations. In the preliminary experiment on 30 selected headwords, this three-stage system achieves a 73% precision rate, a substantial improvement on the 61% achieved using an algorithm we developed earlier based on an improved version of the Smdja's 53% accurate Xtract system.

AB - This paper presents an automatic Chinese collocation extraction system using lexical statistics and syntactical knowledge. This system extracts collocations from manually segmented and tagged Chinese news corpus in three stages. First, the BI-directional BI-Gram statistical measures, including BI-directional strength and spread, and x2 test value, are employed to extract candidate two-word pairs. These candidate word pairs are then used to extract high frequency multi-word collocations from their context. In the third stage, precision is further improved by using syntactical knowledge of collocation patterns between content words to eliminate pseudo collocations. In the preliminary experiment on 30 selected headwords, this three-stage system achieves a 73% precision rate, a substantial improvement on the 61% achieved using an algorithm we developed earlier based on an improved version of the Smdja's 53% accurate Xtract system.

KW - Chinese collocation

KW - Information extraction and statistical models

UR - http://www.scopus.com/inward/record.url?scp=84863860753&partnerID=8YFLogxK

U2 - 10.1109/NLPKE.2003.1275923

DO - 10.1109/NLPKE.2003.1275923

M3 - Conference article published in proceeding or book

SP - 321

EP - 326

BT - NLP-KE 2003 - 2003 International Conference on Natural Language Processing and Knowledge Engineering, Proceedings

PB - IEEE

T2 - International Conference on Natural Language Processing and Knowledge Engineering, NLP-KE 2003

Y2 - 26 October 2003 through 29 October 2003

ER -

An automatic Chinese collocation extraction algorithm based on lexical statistics

Abstract

Conference

Keywords

ASJC Scopus subject areas

More information

Other files and links

Fingerprint

Cite this