• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
搜索

Author:

Liu, Heng (Liu, Heng.) | Wang, Boyue (Wang, Boyue.) | Sun, Yanfeng (Sun, Yanfeng.) | Li, Xiaoyan (Li, Xiaoyan.) | Hu, Yongli (Hu, Yongli.) | Yin, Baocai (Yin, Baocai.) (Scholars:尹宝才)

Indexed by:

CPCI-S EI Scopus

Abstract:

A better knowledge-based visual question answering (KBVQA) model needs to rely on visual features, question features, and related external knowledge to solve an open visual question answering task. Although the existing knowledge-based visual question answering works have achieved some accomplishments, there are still the following challenges: 1) There is a serious lack of visual feature information. Image information is worth a thousand words. Only relying on the converted salient text information is difficult to express the original rich information of the image. 2) The external knowledge acquired is not comprehensive enough, and there is a lack of relevant knowledge directly retrieved by visual feature information. To solve these challenges, we propose a Visual Information-Guided knowledge-based visual question answering (VIG) model. It fully considers the utilization of visual features information. Specifically: 1) We introduce multi-granularity visual information that can comprehensively characterize visual feature information. 2) We consider not only the knowledge retrieved through text information but also the knowledge directly retrieved from visual feature information. Finally, we feed the visual features and retrieved multiple text knowledge into an encoder-decoder module to generate an answer. We perform extensive experiments on the OKVQA dataset and achieve state-of-the-art performance of 60.27% accuracy.

Keyword:

Visual Information-Guided External Knowledge Knowledge-Based VQA

Author Community:

  • [ 1 ] [Liu, Heng]Beijing Univ Technol, Beijing, Peoples R China
  • [ 2 ] [Wang, Boyue]Beijing Univ Technol, Beijing, Peoples R China
  • [ 3 ] [Sun, Yanfeng]Beijing Univ Technol, Beijing, Peoples R China
  • [ 4 ] [Li, Xiaoyan]Beijing Univ Technol, Beijing, Peoples R China
  • [ 5 ] [Hu, Yongli]Beijing Univ Technol, Beijing, Peoples R China
  • [ 6 ] [Yin, Baocai]Beijing Univ Technol, Beijing, Peoples R China

Reprint Author's Address:

  • [Wang, Boyue]Beijing Univ Technol, Beijing, Peoples R China

Show more details

Related Keywords:

Related Article:

Source :

PROCEEDINGS OF THE 2024 27 TH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN, CSCWD 2024

ISSN: 2835-639X

Year: 2024

Page: 1086-1091

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Affiliated Colleges:

Online/Total:503/5293806
Address:BJUT Library(100 Pingleyuan,Chaoyang District,Beijing 100124, China Post Code:100124) Contact Us:010-67392185
Copyright:BJUT Library Technical Support:Beijing Aegean Software Co., Ltd.