• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Luo, Zhiyong (Luo, Zhiyong.) | Song, Rou (Song, Rou.)

收录:

EI Scopus PKU CSCD

摘要:

Disambiguation is one of the most important parts of segment systems in Chinese. A Chinese general-purpose word segmentation (GPWS) system demands higher capacity of disambiguation techniques particularly, because it has functions such as allowing users to create their own dictionaries dynamically and employing multiple user's dictionaries to word segmentation. Based on inspection of the distributions and characteristics of ambiguity fragments (especially overlapping ambiguity fragments) in large-scale real corpus, an improved forward maximum match algorithm for ambiguity fragment detection, as well as a practical 'rules + exceptions' disambiguation strategy, are proposed in this paper. An exhaustive extraction has been made of the overlapping ambiguity sections (about 2.4 million occurrences) from a People's Daily corpus of 100 million characters (234 MB approximately), and open-ended experiments on the above strategy randomly were carried out, which achieved accuracy average of 99%.

关键词:

Algorithms Natural language processing systems Pattern matching Pattern recognition Statistical methods Word processing

作者机构:

  • [ 1 ] [Luo, Zhiyong]College of Computer Science, Beijing University of Technology, Beijing 100022, China
  • [ 2 ] [Luo, Zhiyong]College of Information Science, Beijing Language and Culture University, Beijing 100083, China
  • [ 3 ] [Song, Rou]College of Information Science, Beijing Language and Culture University, Beijing 100083, China

通讯作者信息:

电子邮件地址:

查看成果更多字段

相关关键词:

相关文章:

来源 :

Computer Research and Development

ISSN: 1000-1239

年份: 2006

期: 6

卷: 43

页码: 1122-1128

被引次数:

WoS核心集被引频次: 0

SCOPUS被引频次: 7

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 1

在线人数/总访问数:1058/2907341
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司