In　order　to　effectively　combine　RGB　image　features　with　depth　image　features　for　human　detection,　this　paper　proposes　a　two-stream　RGB-D　human　detection　algorithm　based　on　RFB　network.　The　proposed　algorithm　mainly　contains　three　parts:　RGB-stream,　Depth-stream　and　Channel　Weight　Fusion　(CWF)　strategy.　(1)　The　RGB-stream　extracts　RGB　image　features　using　RFB-Net　as　the　backbone　network.　(2)　By　analyzing　the　results　of　depth　features　visualization,　we　build　the　Depth-stream,　which　can　effectively　extract　the　depth　image　features.　(3)　The　improved　CWF　strategy　can　enhance　the　effectiveness　of　important　channels　in　RGB-D　fusion　features　and　improve　the　capability　of　the　network　expression.　The　experimental　results　show　that　the　proposed　algorithm　has　a　significant　improvement　compared　with　other　algorithms　on　two　common　datasets.