The　paper　aims　to　establish　a　effective　feature　form　of　visual　speech　to　realize　the　Chinese　viseme　recognition.　We　propose　and　discuss　a　representation　model　of　the　visual　speech　which　bases　on　the　local　binary　pattern　(LBP)　and　the　discrete　cosine　transform　(DCT)　of　mouth　images.　The　joint　model　combines　the　advantages　of　the　local　and　global　texture　information　together,　which　shows　better　performance　than　using　the　global　feature　only.　By　computing　LBP　and　DCT　of　each　mouth　frame　capturing　during　the　subject　speaking,　the　Hidden　Markov　Model　(HMM)　is　trained　based　on　the　training　dataset　and　is　employed　to　recognize　the　new　visual　speech.　The　experiments　show　this　visual　speech　feature　model　exhibits　good　performance　in　classifying　the　difference　speaking　states.　©　2012　Springer-Verlag　GmbH.