The　bag　of　visual　words　model　(BoW)　and　its　variants　have　demonstrated　their　effectiveness　for　visual　applications.　The　BoW　model　first　extracts　local　features　and　generates　the　corresponding　codebook　where　the　elements　of　a　codebook　are　viewed　as　visual　words.　However,　the　codebook　is　dataset　dependent　and　has　to　be　generated　for　each　image　dataset.　Besides,　when　we　only　have　a　limited　number　of　training　images,　the　codebook　generated　correspondingly　may　not　be　able　to　encode　images　well.　This　requires　a　lot　of　computational　time　and　weakens　the　generalization　power　of　the　BoW　model.　To　solve　these　problems,　in　this　paper,　we　propose　to　undo　the　dataset　bias　by　linear　codebook　transformation　in　an　unsupervised　manner.　To　represent　each　point　in　the　local　feature　space,　we　need　a　number　of　linearly　independent　basis　vectors.　We　view　the　codebook　as　a　linear　transformation　of　these　basis　vectors.　In　this　way,　we　can　transform　the　pre-learned　codebooks　for　a　new　dataset　using　the　pseudo-inverse　of　the　transformation　matrix.　However,　this　is　an　under-determined　problem　which　may　lead　to　many　solutions.　Besides,　not　all　of　the　visual　words　are　equally　important　for　the　new　dataset.　It　would　be　more　effective　if　we　can　make　some　selection　and　choose　the　discriminative　visual　words　for　transformation.　Specifically,　the　sparsity　constraints　and　the　F-norm　of　the　transformation　matrix　are　used　in　this　paper.　We　propose　an　alternative　optimization　algorithm　to　jointly　search　for　the　optimal　linear　transformation　matrixes　and　the　encoding　parameters.　The　proposed　method　needs　no　labeled　images　from　either　the　source　dataset　or　the　target　dataset.　Image　classification　experimental　results　on　several　image　datasets　show　the　effectiveness　of　the　proposed　method.　(C)　2014　Elsevier　B.V.　All　rights　reserved.