• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Fang, Juan (Fang, Juan.) (学者:方娟) | Wang, Mengxuan (Wang, Mengxuan.) | Sun, Hao (Sun, Hao.)

收录:

EI Scopus

摘要:

With the arrival of big data era, distributed computing framework Hadoop has become the main solution to deal with big data now. People usually promote the performance of distributed computing by adding new computing nodes to cluster. With the expansion of the scale of the cluster, it produces a large amount of power consumption because of lack of reasonable management strategy. So how to make full use of computing resources in the cluster to improve the performance of the whole system and reduce the power consumption has become the main research direction of scholars and industrial circles. For the above, in order to make best use of computing resources and reduce the power consumption, this paper firstly proposes to optimize a reasonable configuration of the parameters provided by Hadoop. Comparing with the default configuration of Hadoop. It shows we can get better performance by parameter tuning. This paper proposes a task scheduling mechanism based on memory usage prediction. In this task schedule, it predicts the future use status of memory in the computing nodes by analyzing the use status before. The task scheduling mechanism can reduce the memory pressure by reducing the allocation of tasks when the computing node is under memory pressure. The task scheduling mechanism can be more flexible by setting the threshold of memory usage. This mechanism based on predicting memory usage can improve the performance of the system by making full use of the computing resources. © Springer Nature Singapore Pte Ltd. 2018.

关键词:

Big data Cluster computing Electric power utilization Forecasting Green computing Industrial research Mobile ad hoc networks Multitasking Sensor networks

作者机构:

  • [ 1 ] [Fang, Juan]Beijing University of Technology, 100 Ping Le Yuan, Chaoyang District, Beijing, China
  • [ 2 ] [Wang, Mengxuan]Beijing University of Technology, 100 Ping Le Yuan, Chaoyang District, Beijing, China
  • [ 3 ] [Sun, Hao]Beijing University of Technology, 100 Ping Le Yuan, Chaoyang District, Beijing, China

通讯作者信息:

  • 方娟

    [fang, juan]beijing university of technology, 100 ping le yuan, chaoyang district, beijing, china

电子邮件地址:

查看成果更多字段

相关关键词:

相关文章:

来源 :

ISSN: 1865-0929

年份: 2018

卷: 747

页码: 227-236

语种: 英文

被引次数:

WoS核心集被引频次: 0

SCOPUS被引频次:

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 2

归属院系:

在线人数/总访问数:732/2900171
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司