We　report　an　ultrafast　optical　switching　device　constructed　by　stacking　two　layers　of　gold　nanowires　into　a　perpendicularly　crossed　network,　which　works　at　a　speed　faster　than　280　fs　with　an　on/off　modulation　depth　of　about　22.4%.　The　two　stacks　play　different　roles　in　enhancing　consistently　the　optical　switching　performance　due　to　their　different　dependence　on　the　polarization　of　optical　electric　fields.　The　cross-plasmon　resonance　based　on　the　interaction　between　the　perpendicularly　stacked　gold　nanowires　and　its　Fano-coupling　with　Rayleigh　anomaly　is　the　dominant　mechanism　for　such　a　high-contrast　optical　switching　device.