The　application　of　deep　learning　methods　in　brain　disease　diagnosis　is　becoming　a　new　research　hotspot.　This　study　constructed　brain　functional　networks　based　on　the　functional　magnetic　resonance　imaging　(fMRI)　data,　and　proposed　a　novel　convolutional　neural　network　combined　with　a　prototype　learning　(CNNPL)　framework　to　classify　brain　functional　networks　for　the　diagnosis　of　autism　spectrum　disorder　(ASD).　At　the　bottom　of　CNNPL,　traditional　CNN　was　employed　as　the　basic　feature　extractor,　while　at　the　top　of　CNNPL　multiple　prototypes　were　automatically　learnt　on　the　features　to　represent　different　categories.　A　generalized　prototype　loss　based　on　distance　cross-entropy　was　proposed　to　jointly　learn　the　parameters　of　the　CNN　feature　extractor　and　the　prototypes.　The　classification　was　implemented　with　prototype　matching.　A　transfer　learning　strategy　was　introduced　to　our　CNNPL　for　weight　initialization　in　the　subsequent　fine-tuning　phase　to　promote　model　training.　We　conducted　systematic　experiments　on　the　aggregate　multi-sites　ASD　dataset.　Experimental　results　revealed　that　our　model　outperforms　the　current　state-of-the-art　methods　in　ASD　classification　and　can　reliably　learn　inter-site　biomarkers,　indicating　the　robustness　of　our　model　on　large-scale　dataset　with　inter-site　variability.　Furthermore,　our　model　demonstrated　robust　learning　capability　for　high-level　organization　of　brain　functionality.　Our　study　also　identified　important　brain　regions　as　biomarkers　associated　with　ASD　classification.　Together,　our　proposed　model　provides　a　promising　solution　for　learning　and　classifying　brain　functional　networks,　and　thus　contributes　to　the　biomarker　extraction　and　imaging　diagnosis　of　ASD.