In　this　paper,　we　use　mid-level　features　to　solve　the　problems　of　indoor　scene　image　classification.　The　mid-level　patches　should　satisfy　two　conditions:　(1)　representative,　they　should　occur　frequently　enough　in　the　visual　world;　(2)　discriminative,　they　need　to　be　different　enough　from　the　rest　of　the　visual　world.　In　this　paper,　we　propose　a　method　to　select　the　initial　patches.　It　can　eliminate　a　large　number　of　patches　which　are　mismatch　the　conditions,　and　there　is　no　need　manual　processing.　For　initial　patches　we　adopt　unsupervised　cluster　algorithm　on　HOG　space.　Then,　using　the　purity-discriminative　evaluation　criteria,　the　top　r　clusters　were　selected　to　represent　each　scene.　The　experimental　results　on　MIT　Indoor　67　scene　image　classification　datasets　indicate　that　our　method　can　achieve　very　promising　performance.