Though　action　recognition　based　on　complete　videos　has　achieved　great　success　recently,　action　prediction　remains　a　challenging　task　as　the　information　provided　by　partial　videos　is　not　discriminative　enough　for　classifying　actions.　In　this　paper,　we　propose　a　Deep　Residual　Feature　Learning　(DeepRFL)　framework　to　explore　more　discriminative　information　from　partial　videos,　achieving　similar　representations　as　those　of　complete　videos.　The　proposed　method　is　based　on　residual　learning,　which　captures　the　salient　differences　between　partial　videos　and　their　corresponding　full　videos.　The　partial　videos　can　attain　the　missing　information　by　learning　from　features　of　complete　videos　and　thus　improve　the　discriminative　power.　Moreover,　our　model　can　be　trained　efficiently　in　an　end-to-end　fashion.　Extensive　evaluations　on　the　challenging　UCF101　and　HMDB51　datasets　demonstrate　that　the　proposed　method　outperforms　state-of-the-art　results.