Feature　plays　an　important　role　in　pedestrian　detection,　and　considerable　progress　has　been　made　on　shape-based　descriptors.　However,　color　cues　have　barely　been　devoted　to　detection　tasks,　seemingly　due　to　the　variable　appearance　of　pedestrians.　In　this　paper,　Color　Maximal-Dissimilarity　Pattern　(CMDP)　is　proposed　to　encode　color　cues　by　two　core　operations,　i.e.,　oriented　filtering　and　max-pooling,　which　emulate　the　functions　of　primary　visual　cortex　(V1).　The　extensively　experimental　results　reveal　that　the　biologically-explainable　encoding　scheme　increases　the　invariance　of　color　cues,　and　outperforms　the　state-of-the-art　color　descriptor　in　terms　of　both　accuracy　and　speed.