Alert button
Picture for Zhikun Yang

Zhikun Yang

Alert button

Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization

Add code
Bookmark button
Alert button
Jul 28, 2019
Weisen Wang, Zhiyan Xu, Weihong Yu, Jianchun Zhao, Jingyuan Yang, Feng He, Zhikun Yang, Di Chen, Dayong Ding, Youxin Chen, Xirong Li

Figure 1 for Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization
Figure 2 for Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization
Figure 3 for Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization
Figure 4 for Two-Stream CNN with Loose Pair Training for Multi-modal AMD Categorization

This paper studies automated categorization of age-related macular degeneration (AMD) given a multi-modal input, which consists of a color fundus image and an optical coherence tomography (OCT) image from a specific eye. Previous work uses a traditional method, comprised of feature extraction and classifier training that cannot be optimized jointly. By contrast, we propose a two-stream convolutional neural network (CNN) that is end-to-end. The CNN's fusion layer is tailored to the need of fusing information from the fundus and OCT streams. For generating more multi-modal training instances, we introduce Loose Pair training, where a fundus image and an OCT image are paired based on class labels rather than eyes. Moreover, for a visual interpretation of how the individual modalities make contributions, we extend the class activation mapping technique to the multi-modal scenario. Experiments on a real-world dataset collected from an outpatient clinic justify the viability of our proposal for multi-modal AMD categorization.

* accepted by MICCAI 2019 
Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon