In　the　control　algorithm　of　autopilot　system,　the　Deep　Learning　method　plays　a　vital　role.　Since　the　convolutional　neural　network　(CNN)　model　used　in　automatic　driving　has　a　huge　amount　of　parameters　and　the　training　results　are　prone　to　overfitting,　an　excellent　model　is　necessary.　In　this　paper,　an　end-to-end　control　method　was　proposed　to　apply　a　convolutional　neural　network　with　a　new　network　structure　to　control　the　steering　angle　and　speed　of　the　vehicle　and　reach　the　goal　of　automatic　vehicle　driving.　The　experimental　results　show　that　it　not　only　greatly　reduces　the　number　of　parameters,　but　also　keeps　the　error　rate　of　the　experimental　results　at　the　low　level.　©　2019　IEEE.