The　bag-of-visual-words　model　plays　a　very　important　role　for　visual　applications.　Local　features　are　first　extracted　and　then　encoded　to　get　the　histogram-based　image　representation.　To　encode　local　features,　a　proper　codebook　is　needed.　Usually,　the　codebook　has　to　be　generated　for　each　data　set　which　means　the　codebook　is　data　set　dependent.　Besides,　the　codebook　may　be　biased　when　we　only　have　a　limited　number　of　training　images.　Moreover,　the　codebook　has　to　be　pre-learned　which　cannot　be　updated　quickly,　especially　when　applied　for　online　visual　applications.　To　solve　the　problems　mentioned　above,　in　this　paper,　we　propose　a　novel　implicit　codebook　transfer　method　for　visual　representation.　Instead　of　explicitly　generating　the　codebook　for　the　new　data　set,　we　try　to　make　use　of　pre-learned　codebooks　using　non-linear　transfer.　This　is　achieved　by　transferring　the　pre-learned　codebooks　with　non-linear　transformation　and　use　them　to　reconstruct　local　features　with　sparsity　constraints.　The　codebook　does　not　need　to　be　explicitly　generated　but　can　be　implicitly　transferred.　In　this　way,　we　are　able　to　make　use　of　pre-learned　codebooks　for　new　visual　applications　by　implicitly　learning　the　codebook　and　the　corresponding　encoding　parameters　for　image　representation.　We　apply　the　proposed　method　for　image　classification　and　evaluate　the　performance　on　several　public　image　data　sets.　Experimental　results　demonstrate　the　effectiveness　and　efficiency　of　the　proposed　method.