As　the　fast　developing　of　computer　vision,　pedestrian　detection　has　become　a　hot　topic　of　current　research.　Accurate　and　real-Time　detection　of　pedestrians　from　video　sequences　is　of　great　practical　importance　in　the　field　of　intelligent　video　surveillance.　Currently,　most　pedestrian　detection　systems　are　built　on　RGB　images.　However,　RGB　images　are　easily　affected　by　light　changes　and　weather　conditions.　In　low-light　scenes　the　quality　is　poor.　Infrared　information,　however,　relies　only　on　the　thermal　radiation　of　the　object　and　can　clearly　show　the　outline　of　the　pedestrian.　This　paper　therefore　proposes　a　multispectral　based　pedestrian　detection　algorithm　(multispectral　images　include　RGB　and　IR　images).　Specifically,　firstly,　design　an　asymmetric　two-stream　network　structure　to　extract　RGB　image　and　infrared　image　features　respectively.　Then　a　multimodal　fusion　mechanism　is　proposed　to　fuse　RGB　feature　maps　and　IR　feature　maps　for　channel　fusion.　Experimental　results　demonstrate　that　our　method　improves　the　target　detection　properties.　The　map　value　reaches　0.955　on　the　dataset　GTOT　and　the　own　proposed　dataset　School.　©　2022　IEEE.