Extreme　weather　always　brings　potential　risk　to　driving,　which　leads　to　people＇s　life　and　property　being　put　into　great　dangers.　Therefore,　the　automatic　recognition　of　extreme　weather　plays　an　important　role　in　the　application　of　the　highway　traffic　condition　warning,　automobile　auxiliary　driving,　climate　analysis　and　so　on.　Generally,　multiple　sensors　are　adopted　in　traditional　methods　of　automatic　extreme　weather　recognition　with　artificial　participation　and　low　accuracy.　A　new　extreme　weather　recognition　method　based　on　images　by　using　computer　vision　manners　has　been　proposed　in　this　paper.　Since　the　weather　is　affected　by　many　factors,　features　that　can　accurately　represent　various　weather　characteristics　are　difficult　to　be　extracted.　Therefore,　in　this　paper,　convolutional　neural　networks　(CNNs)　are　applied　to　settle　this　problem.　Features　of　extreme　weather　and　recognition　models　are　generated　from　big　data.　Moreover,　a　large-scale　extreme　weather　dataset,　＂WeatherDataset＂,　has　been　collected,　in　which　16635　extreme　weather　images　are　divided　into　four　classes　(sunny,　rainstorm,　blizzard,　and　fog),　and　complex　scenes　are　coverd.　A　recognition　model　for　extreme　weather　is　obtained　through　two　steps:　Pre-training　and　Fine　Tuning.　In　Pre-training　step,　ILSVRC-2012　Dataset　is　trained　to　obtain　the　model　of　ILSVRC　using　GoogLeNet.　A　more　accurate　model　for　extreme　weather　recognition　is　obatined　by　further　fine-tuning　GoogLeNet　on　WeatherDataset.　The　experimental　results　show　that　the　proposed　method　is　able　to　achieve　a　high　performance　with　the　recognition　accuracy　rate　of　94.5%　and　can　meet　the　requirements　of　some　real　applications.