In　this　paper,　we　propose　a　novel　visual　tracking　algorithm　by　combining　the　structure-aware　network　(SA-Net)　and　spatial-temporal　regression　model.　We　first　use　SA-Net　to　obtain　the　initial　location　proposal,　and　the　deep　features　are　extracted　using　a　fine-tuned　convolutional　neural　network　model.　Finally,　both　the　location　proposal　and　deep　features,　including　historical　information,　are　input　into　the　long　short-term　memory　(LSTM)　for　end-to-end　spatial　temporal　regression　to　adjust　the　initial　location　proposal　from　SA-Net.　The　experimental　results　on　the　challenging　OTB　dataset　demonstrate　that　the　proposed　scheme　is　robust　to　missing　tracking　caused　by　occlusion　or　object　deformation.　Additionally,　the　compared　experiments　show　that　the　proposed　scheme　is　more　competitive　than　state-of-the-art　algorithms.　©　2018　IEEE.