• 综合
  • 标题
  • 关键词
  • 摘要
  • 学者
  • 期刊-刊名
  • 期刊-ISSN
  • 会议名称
搜索

作者:

Xie, Bochen (Xie, Bochen.) | Deng, Yongjian (Deng, Yongjian.) | Shao, Zhanpeng (Shao, Zhanpeng.) | Li, Youfu (Li, Youfu.)

收录:

EI Scopus SCIE

摘要:

Bio-inspired event cameras record a scene as sparse and asynchronous "events" by detecting per-pixel brightness changes. Such cameras show great potential in challenging scene understanding tasks, benefiting from the imaging advantages of high dynamic range and high temporal resolution. Considering the complementarity between event and standard cameras, we propose a multi-modal fusion network (EISNet) to improve the semantic segmentation performance. The key challenges of this topic lie in (i) how to encode event data to represent accurate scene information and (ii) how to fuse multi-modal complementary features by considering the characteristics of two modalities. To solve the first challenge, we propose an Activity-Aware Event Integration Module (AEIM) to convert event data into frame-based representations with high-confidence details via scene activity modeling. To tackle the second challenge, we introduce the Modality Recalibration and Fusion Module (MRFM) to recalibrate modal-specific representations and then aggregate multi-modal features at multiple stages. MRFM learns to generate modal-oriented masks to guide the merging of complementary features, achieving adaptive fusion. Based on these two core designs, our proposed EISNet adopts an encoder-decoder transformer architecture for accurate semantic segmentation using events and images. Experimental results show that our model outperforms state-of-the-art methods by a large margin on event-based semantic segmentation datasets.

关键词:

Noise measurement multi-modal fusion attention mechanism Visualization Semantic segmentation Cameras Event camera Standards Semantics semantic segmentation Task analysis

作者机构:

  • [ 1 ] [Xie, Bochen]City Univ Hong Kong, Dept Mech Engn, Hong Kong, Peoples R China
  • [ 2 ] [Li, Youfu]City Univ Hong Kong, Dept Mech Engn, Hong Kong, Peoples R China
  • [ 3 ] [Deng, Yongjian]Beijing Univ Technol, Coll Comp Sci, Beijing 100124, Peoples R China
  • [ 4 ] [Shao, Zhanpeng]Hunan Normal Univ, Coll Informat Sci & Engn, Changsha 410081, Peoples R China

通讯作者信息:

查看成果更多字段

相关关键词:

相关文章:

来源 :

IEEE TRANSACTIONS ON MULTIMEDIA

ISSN: 1520-9210

年份: 2024

卷: 26

页码: 8639-8650

7 . 3 0 0

JCR@2022

被引次数:

WoS核心集被引频次:

SCOPUS被引频次: 6

ESI高被引论文在榜: 0 展开所有

万方被引频次:

中文被引频次:

近30日浏览量: 1

归属院系:

在线人数/总访问数:423/4940289
地址:北京工业大学图书馆(北京市朝阳区平乐园100号 邮编:100124) 联系我们:010-67392185
版权所有:北京工业大学图书馆 站点建设与维护:北京爱琴海乐之技术有限公司