With　the　development　of　visual　social　network,　more　and　more　people　like　to　present　themselves　using　images　or　videos.　Visual　emotion　analysis　is　becoming　one　of　hot　research　topics.　According　to　Jou’s　idea [1],　emotion　presentations　of　the　Western　and　the　Eastern　are　much　different　due　to　the　culture　difference.　There　are　some　popular　emotion　models　such　as　Plutchik’s　model　and　so　on.　And　there　is　not　one-to-one　correspondence　between　these　emotion　models.　All　of　the　existing　image　databases　for　emotion　analysis　are　built　by　Plutchik’s　and　Mikels’s　emotion　models.　However,　most　researches　on　Chinese　text　emotion　analysis　used　Xu’s　model[2].　And　there　are　not　the　corresponding　image　datasets　for　emotion　analysis　with　Xu’s　model.　In　this　paper　we　establish　an　image　dataset　for　emotion　analysis　by　collecting　images　from　Flickr　using　Chinese　Emotion　Ontology　of　Xu’s　model.　In　addition,　we　design　a　dataset　refinement　(de-noising)　strategy　to　promote　the　confidence　of　emotion　labels　for　the　images.　Finally,　we　establish　the　dataset　CH-EmoD　which　includes　a　sub　dataset　with　single　emotion　label　and　a　sub　dataset　with　multiple　emotion　labels.　Furthermore,　we　provide　the　baselines　of　emotion　classification　and　multi-label　emotion　classification　by　using　state-of-the-art　emotion/sentiment　classifications　algorithms　Alexnet　[3]　and　PCNN　[4].　The　experimental　results　demonstrate　that　the　dataset　works　well　on　emotion　classification　and　multi-label　emotion　classification　and　the　proposed　dataset　refinement　strategy　is　effective.　©　Springer　Nature　Switzerland　AG　2018.