The　goal　of　sign　language　recognition　(SLR)　is　to　translate　the　sign　language　into　text,　and　provide　a　convenient　tool　for　the　communication　between　the　deaf-mute　and　the　ordinary.　In　this　paper,　we　formulate　an　appropriate　model　based　on　convolutional　neural　network　(CNN)　combined　with　Long　Short-Term　Memory　(LSTM)　network,　in　order　to　accomplish　the　continuous　recognition　work.　With　the　strong　ability　of　CNN,　the　information　of　pictures　captured　from　Chinese　sign　language　(CSL)　videos　can　be　learned　and　transformed　into　vector.　Since　the　video　can　be　regarded　as　an　ordered　sequence　of　frames,　LSTM　model　is　employed　to　connect　with　the　fully-connected　layer　of　CNN.　As　a　recurrent　neural　network　(RNN),　it　is　suitable　for　sequence　learning　tasks　with　the　capability　of　recognizing　patterns　defined　by　temporal　distance.　Compared　with　traditional　RNN,　LSTM　has　performed　better　on　storing　and　accessing　information.　We　evaluate　this　method　on　our　self-built　dataset　including　40　daily　vocabularies.　The　experimental　results　show　that　the　recognition　method　with　CNN-LSTM　can　achieve　a　high　recognition　rate　with　small　training　sets,　which　will　meet　the　needs　of　real-time　SLR　system.