Depth　maps　obtained　by　the　RGB-D　camera　common　contain　noise　in　addition　to　the　general　Gaussian　noise　such　as　uncertain　pixels,　incredible　singular　pixels,　as　well　as　some　black　areas　composed　of　pixels　with　no　depth　values.　In　this　paper,　we　purpose　a　depth　image　restoration　model　based　on　the　sparse　representation　theory　to　enhance　the　quality　of　depth　maps.　To　this　model,　we　consider　the　non-local　similarity　and　singular　pixels　of　depth　map　as　constraints　and　fusion　detail　information　of　the　corresponding　high　resolution　color　image.　Eventually,　the　restoration　for　depth　map　is　converted　into　the　optimization　model　problem,　and　the　process　the　optimization　solution　is　also　given.　Finally,　several　experiments　on　public　dataset　demonstrated　that　the　purposed　model　is　validity　for　depth　images　enhancement.　©　2016,　UK　Simulation　Society.　All　rights　reserved.