Cataract　is　the　most　prevailing　reason　for　blindness　across　the　globe,　which　occupies　about　4.2%　population　of　the　world.　Even　with　the　developments　in　visual　sciences,　fundus　image-based　diagnosis　is　deemed　as　a　gold　standard　for　cataract　detection　and　grading.　Though　the　increase　in　the　workload　of　ophthalmologists　and　complexity　of　fundus　images,　the　results　may　be　subject　to　intelligence.　Therefore,　the　development　of　an　automatic　method　for　cataract　detection　is　necessary　to　prevent　visual　impairment　and　save　medical　resources.　This　paper　aims　to　provide　a　novel　hybrid　convolutional　and　recurrent　neural　network　(CRNN)　for　fundus　image-based　cataract　classification.　The　proposed　CRNN　fuses　the　advantages　of　convolution　neural　network　and　recurrent　neural　network　to　preserve　long-　and　short-term　spatial　correlation　between　the　patches.　Coupled　with　transfer　learning,　we　adopt　AlexNet,　GoogLeNet,　ResNet　and　VGGNet　to　extract　multilevel　feature　representation　and　to　analyse　how　well　these　models　perform　cataract　classification.　The　results　demonstrate　that　the　proposed　method　outperforms　state-of-the-art　methods　with　an　average　accuracy　of　0.9739　for　four-class　cataract　classification　and　provides　a　compelling　reason　to　be　applied　for　other　retinal　diseases.