Depth　cameras　have　gained　significant　popularity　due　to　their　affordable　cost　in　recent　years.　However,　the　resolution　of　depth　map　captured　by　these　cameras　is　rather　limited,　and　thus　it　hardly　can　be　directly　used　in　visual　depth　perception　and　3D　reconstruction.　In　order　to　handle　this　problem,　we　propose　a　novel　multiclass　dictionary　learning　method,　in　which　depth　image　is　divided　into　classified　patches　according　to　their　geometrical　directions　and　a　sparse　dictionary　is　trained　within　each　class.　Different　from　previous　SR　works,　we　build　the　correspondence　between　training　samples　and　their　corresponding　register　color　image　via　sparse　representation.　We　further　use　the　adaptive　autoregressive　model　as　a　reconstruction　constraint　to　preserve　smooth　regions　and　sharp　edges.　Experimental　results　demonstrate　that　our　method　outperforms　state-of-the-art　methods　in　depth　map　super-resolution　in　terms　of　both　subjective　quality　and　objective　quality.　©　2017　IEEE.