The　good　fusion　of　multi-scale　features　obtained　by　Convolutional　neural　networks　(CNNs)　is　key　to　semantic　edge　detection;　however,　obtaining　fusion　is　challenging.　This　paper　presents　a　Multi-scale　Spatial　Context-based　deep　network　for　Semantic　Edge　Detection　(MSC-SED).　Different　from　state-of-the-art　methods,　MSC-SED　gradually　fuses　multi-scale　low-to-high　level　CNN　features　in　an　end-to-end　architecture.　This　fusion　structure　obtains　rich　multi-scale　features　while　enhancing　the　details　of　higher-level　features.　Beside　the　overall　structure,　we　present　the　following　two　functional　modules:　the　Context　Aggregation　Module　(CAM)　and　Location-Aware　fusion　Module　(LAM).　The　CAM　helps　to　enrich　context　in　features　at　each　stage,　before　and　after　fusion.　The　LAM　helps　to　selectively　integrate　lower-level　features.　The　proposed　method　outperforms　state-of-the-art　approaches　in　terms　of　both　the　edge　quality　and　the　accuracy　of　edge　categorization　on　both　the　SBD　and　Cityscapes　datasets.