• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Liang, Yi (Liang, Yi.) | Li, Guangrui (Li, Guangrui.) | Wang, Lei (Wang, Lei.) | Hu, Yanpeng (Hu, Yanpeng.)

收录:

EI Scopus

摘要:

Map/reduce is a popular parallel processing framework for massive-scale data-intensive computing. The data-iterative application is composed of a serials of map/reduce jobs and need to repeatedly process some data files among these jobs. The existing implementation of map/reduce framework focus on perform data processing in a single pass with one map/reduce job and do not directly support the data-iterative applications, particularly in term of the explicit specification of the repeatedly processed data among jobs. In this paper, we propose an extended version of Hadoop map/reduce framework called Dacoop. Dacoop extends Map/Reduce programming interface to specify the repeatedly processed data, introduces the shared memorybased data cache mechanism to cache the data since its first access, and adopts the caching-aware task scheduling so that the cached data can be shared among the map/reduce jobs of data-iterative applications. We evaluate Dacoop on two typical data-iterative applications: k-means clustering and the domain rule reasoning in sementic web, with real and synthetic datasets. Experimental results show that the data-iterative applications can gain better performance on Dacoop than that on Hadoop. The turnaround time of a data-iterative application can be reduced by the maximum of 15.1%. © 2011 IEEE.

关键词:

Multitasking Distributed computer systems Scheduling algorithms K-means clustering Cache memory

作者机构:

  • [ 1 ] [Liang, Yi]Department of Computer Science, Beijing University of Technology, Beijing, China
  • [ 2 ] [Li, Guangrui]Department of Computer Science, Beijing University of Technology, Beijing, China
  • [ 3 ] [Wang, Lei]Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
  • [ 4 ] [Hu, Yanpeng]Hwellzen Software Center, Shanghai, China

通讯作者信息:

电子邮件地址:

查看成果更多字段

相关关键词:

相关文章:

来源 :

年份: 2011

页码: 207-214

语种: 英文

被引次数:

WoS核心集被引频次:

SCOPUS被引频次: 2

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 0

在线人数/总访问数:212/4520380
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司