Instance　semantic　segmentation　has　already　been　a　promising　direction,　but　many　leading　approaches　are　lack　of　detailed　structural　information　and　unable　to　segment　small　size　objects.　In　this　paper,　we　present　a　novel　segmentation　framework,　called　Scale-aware　Patch　Fusion　Network　(SPF).　Our　unified　end-to-end　trainable　network　consists　of　three　components,　namely,　multi-scale　patch　generator,　semantic　segmentation　network　and　patch　fusion　algorithm.　This　patchbased　method　aggregates　information　from　different　scales　of　patches　via　fusing　local　segmentation　prediction　results.　The　proposed　approach　is　thus　more　effective　and　simple.　Experiments　on　VOC　2012　segmentation　val,　VOC　2012　SDS　val,　MS　COCO　datasets　validate　the　effectiveness　of　our　approach.