• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Pan, Guangyuan (Pan, Guangyuan.) | Qiao, Junfei (Qiao, Junfei.) (Scholars:乔俊飞) | Chai, Wei (Chai, Wei.) | Dimopoulos, Nikitas (Dimopoulos, Nikitas.)

Indexed by:

EI Scopus

Abstract:

Restricted Boltzmann Machine is a fundamental method in deep learning networks. Training and generalization is an ill-defined problem in that many different networks may achieve the training goal; however each will respond differently to an unknown input. Traditional approaches include stopping the training early and/or restricting the size of the network These approaches ameliorate the problem of over-fitting where the network learns the patterns presented but is unable to generalize. Bayesian regularization addresses these issues by requiring the weights of the network to attain a minimum magnitude. This ensures that non-contributing weights are reduced significantly and the resulting network represents the essence of the inter-relations of the training. Bayesian Regularization simply introduces an additional term to the objective function. This term comprises the sum of the squares of the weights. The optimization process therefore not only achieves the objective of the original cost (i.e. the minimization of an error metric) but it also ensures that this objective is achieved with minimum-magnitude weights. We have introduced Bayesian Regularization in the training of Restricted Boltzmann Machines and have applied this method in experiments of hand-written numbers classification. Our experiments showed that by adding Bayesian regularization in the training of RBMs, we were able to improve the generalization capabilities of the trained network by reducing its recognition errors by more than 1.6%. © 2014 IEEE.

Keyword:

Deep learning Classification (of information) Neural networks

Author Community:

  • [ 1 ] [Pan, Guangyuan]Institute of Intelligent Systems, Beijing University of Technology, NO.100 Pingleyuan, ChaoYang District, Beijing; 100124, China
  • [ 2 ] [Qiao, Junfei]Institute of Intelligent Systems, Beijing University of Technology, NO.100 Pingleyuan, ChaoYang District, Beijing; 100124, China
  • [ 3 ] [Chai, Wei]Institute of Intelligent Systems, Beijing University of Technology, NO.100 Pingleyuan, ChaoYang District, Beijing; 100124, China
  • [ 4 ] [Dimopoulos, Nikitas]Department of Electrical and Computer Engineering, University of Victoria, Victoria; BC, Canada

Reprint Author's Address:

  • [pan, guangyuan]institute of intelligent systems, beijing university of technology, no.100 pingleyuan, chaoyang district, beijing; 100124, china

Email:

Show more details

Related Keywords:

Related Article:

Source :

Year: 2014

Page: 2935-2939

Language: English

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count: 12

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Affiliated Colleges:

Online/Total:620/5311753
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.