Sparse　Representation-based　Classifier　(SRC)　is　less　sensitive　to　the　shortage　of　data　and　the　selection　of　feature　space.　In　this　paper,　SRC　is　adopted　to　perform　automatic　analysis　of　tongue　substance　color　and　coating　color　which　is　considered　as　small　dataset　classification　task.　Firstly,　for　both　training　samples　and　testing　samples,　the　tongue　body　regions　are　segmented,　the　regions　of　tongue　substance　and　coating　are　separated　and　the　features　of　these　regions　are　extracted.　Then,　the　feature　vectors　of　testing　samples　are　represented　by　sparse　coding.　Finally,　the　feature　vectors　of　testing　samples　are　reconstructed　for　each　category,　the　residuals　are　calculated　and　the　category　of　the　smallest　residual　is　determined　as　the　classification　result.　Experimental　results　show　that　the　proposed　method　can　achieve　the　classification　accuracy　of　87.80%　and　90.51%　for　the　colors　of　tongue　substance　and　tongue　coating　respectively.　In　addition,　the　classification　accuracy　of　tongue　coating　color　can　reach　to　91.19%　with　the　assistance　of　visual　samples　by　SRC.　The　proposed　method　provides　a　new　method　for　automatic　analysis　of　tongue　color,　and　it　is　also　a　new　application　of　SRC.