Automatic　annotating　images　by　equipment　is　of　great　interest　as　it　meets　one＇s　common　need　for　retrieving　image　content.　Usually　image　content　description　with　keywords　is　regarded　as　a　visual-word　correlation　process.　However,　in　view　of　the　viewer＇s　psychology,　image　to　words　is　a　kind　of　cognition　process,　which　depends　more　on　the　experience　for　one　to　understand　what＇s　in　an　image.　In　this　paper,　we　introduce　a　semantic　vocabulary　cognition　model　to　improve　the　image　annotation　result.　In　the　training　process,　images　are　annotated　using　common　probability　model　that　computes　the　correlation　between　images　and　the　keywords.　Then　a　semantic　vocabulary　topic　is　computed　and　compared　with　the　words　correlation　described　in　WordNet.　Finally　the　divergence　of　the　two　distribution　is　computed　to　remove　the　irrational　annotations.　Experimental　results　show　that　the　annotation　results　are　improved　through　this　model.