• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Chen, Jilan (Chen, Jilan.) | Wang, Dan (Wang, Dan.) (学者:王丹) | Fu, Lihua (Fu, Lihua.) | Zhao, Wenbing (Zhao, Wenbing.)

收录:

EI Scopus

摘要:

Hadoop distributed file system (HDFS) has been widely used in various clusters to build large scale and high performance systems. However, it is designed to mainly handle big size files, therefore the performance processing massive small files is relatively low because of huge numbers of small files imposing heavy burden on Namenode of HDFS. Focusing the problem about HDFS when processing small files, an approach to improve I/O performance of small files on HDFS is introduced. Our main idea is to merge small files in the same directory into large one and accordingly build index for each small file to enhance storage efficiency of small files and reduce burden on Namenode caused by metadata. Furthermore, a kind of Cache strategy to improve the reading efficiency of small files on HDFS is presented. Relevant design, data structure and implementation are described. The experimental results indicate that the method proposed can improve the efficiency of processing massive small files on HDFS.

关键词:

Digital storage Efficiency File organization Processing

作者机构:

  • [ 1 ] [Chen, Jilan]Beijing University of Technology, China
  • [ 2 ] [Wang, Dan]Beijing University of Technology, China
  • [ 3 ] [Fu, Lihua]Beijing University of Technology, China
  • [ 4 ] [Zhao, Wenbing]Beijing University of Technology, China

通讯作者信息:

电子邮件地址:

查看成果更多字段

相关关键词:

来源 :

International Journal of Digital Content Technology and its Applications

ISSN: 1975-9339

年份: 2012

期: 20

卷: 6

页码: 296-304

被引次数:

WoS核心集被引频次: 0

SCOPUS被引频次: 13

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 2

归属院系:

在线人数/总访问数:3473/2974949
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司