• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Liu, Heng (Liu, Heng.) | Wang, Boyue (Wang, Boyue.) | Sun, Yanfeng (Sun, Yanfeng.) | Li, Xiaoyan (Li, Xiaoyan.) | Hu, Yongli (Hu, Yongli.) | Yin, Baocai (Yin, Baocai.) (学者:尹宝才)

收录:

CPCI-S EI Scopus

摘要:

A better knowledge-based visual question answering (KBVQA) model needs to rely on visual features, question features, and related external knowledge to solve an open visual question answering task. Although the existing knowledge-based visual question answering works have achieved some accomplishments, there are still the following challenges: 1) There is a serious lack of visual feature information. Image information is worth a thousand words. Only relying on the converted salient text information is difficult to express the original rich information of the image. 2) The external knowledge acquired is not comprehensive enough, and there is a lack of relevant knowledge directly retrieved by visual feature information. To solve these challenges, we propose a Visual Information-Guided knowledge-based visual question answering (VIG) model. It fully considers the utilization of visual features information. Specifically: 1) We introduce multi-granularity visual information that can comprehensively characterize visual feature information. 2) We consider not only the knowledge retrieved through text information but also the knowledge directly retrieved from visual feature information. Finally, we feed the visual features and retrieved multiple text knowledge into an encoder-decoder module to generate an answer. We perform extensive experiments on the OKVQA dataset and achieve state-of-the-art performance of 60.27% accuracy.

关键词:

Visual Information-Guided External Knowledge Knowledge-Based VQA

作者机构:

  • [ 1 ] [Liu, Heng]Beijing Univ Technol, Beijing, Peoples R China
  • [ 2 ] [Wang, Boyue]Beijing Univ Technol, Beijing, Peoples R China
  • [ 3 ] [Sun, Yanfeng]Beijing Univ Technol, Beijing, Peoples R China
  • [ 4 ] [Li, Xiaoyan]Beijing Univ Technol, Beijing, Peoples R China
  • [ 5 ] [Hu, Yongli]Beijing Univ Technol, Beijing, Peoples R China
  • [ 6 ] [Yin, Baocai]Beijing Univ Technol, Beijing, Peoples R China

通讯作者信息:

  • [Wang, Boyue]Beijing Univ Technol, Beijing, Peoples R China

查看成果更多字段

相关关键词:

相关文章:

来源 :

PROCEEDINGS OF THE 2024 27 TH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN, CSCWD 2024

ISSN: 2835-639X

年份: 2024

页码: 1086-1091

被引次数:

WoS核心集被引频次:

SCOPUS被引频次:

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 1

归属院系:

在线人数/总访问数:622/4948620
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司