• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Zhang, Xiaodan (Zhang, Xiaodan.) | Yang, Sisi (Yang, Sisi.) | Shi, Yanzhao (Shi, Yanzhao.) | Ji, Junzhong (Ji, Junzhong.) (学者:冀俊忠) | Liu, Ying (Liu, Ying.) | Wang, Zheng (Wang, Zheng.) | Xu, Huimin (Xu, Huimin.)

收录:

EI Scopus SCIE

摘要:

Brain Computed Tomography (CT) report generation, which aims to assist radiologists in diagnosing cerebrovascular diseases efficiently, is challenging in feature representation for dozens of images and language descriptions with several sentences. Existing report generation methods have achieved significant achievement based on the encoder-decoder framework and attention mechanism. However, current research has limitations in solving the many-to-many alignment between the multi-images of Brain CT imaging and the multi-sentences of Brain CT report, and fails to attend to critical images and lesion areas, resulting in inaccurate descriptions. In this paper, we propose a novel Weakly Guided Attention Model with Hierarchical Interaction, named WGAM-HI, to improve Brain CT report generation. Specifically, WGAM-HI conducts many-to-many matching for multiple visual images and semantic sentences via a hierarchical interaction framework with a two -layer attention model and a two-layer report generator. In addition, two weakly guided mechanisms are proposed to facilitate the attention model to focus more on important images and lesion areas under the guidance of pathological events and Gradient-weighted Class Activation Mapping (Grad-CAM) respectively. The pathological event acts as a bridge between the essential serial images and the corresponding sentence, and the Grad-CAM bridges the lesion areas and pathology words. Therefore, under the hierarchical interaction with the weakly guided attention model, the report generator generates more accurate words and sentences. Experiments on the Brain CT dataset demonstrate the effectiveness of WGAM-HI in attending to important images and lesion areas gradually, and generating more accurate reports.

关键词:

Brain CT Hierarchical interaction Weakly guided attention Medical report generation

作者机构:

  • [ 1 ] [Zhang, Xiaodan]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 2 ] [Yang, Sisi]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 3 ] [Shi, Yanzhao]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 4 ] [Ji, Junzhong]Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
  • [ 5 ] [Liu, Ying]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China
  • [ 6 ] [Wang, Zheng]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China
  • [ 7 ] [Xu, Huimin]Peking Univ Third Hosp, Dept Radiol, Beijing, Peoples R China

通讯作者信息:

查看成果更多字段

相关关键词:

来源 :

COMPUTERS IN BIOLOGY AND MEDICINE

ISSN: 0010-4825

年份: 2023

卷: 167

7 . 7 0 0

JCR@2022

被引次数:

WoS核心集被引频次:

SCOPUS被引频次: 9

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 0

归属院系:

在线人数/总访问数:336/4948013
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司