This　paper　is　on　scene　recognition　for　a　light　railway　vehicle　safety　system　using　a　new　patch-based　approach　for　key　frame　identification.　The　approach　is　different　from　those　conventional　approaches　using　for　example　SIFT,　SURF,　BRIEF,　or　ORB　for　individual　frame　recognition.　We　propose　a　new　unsupervised　and　learning-based　key　region　detection　method.　The　proposed　method　contains　two　parts.　In　the　offline　part,　the　key　regions　with　discriminative　information　are　identified　from　single　reference　sequence　captured　by　monocular　camera　with　unsupervised　method.　The　discrimination　power　for　a　region　is　defined　as　the　difference　between　this　region　and　all　other　regions　in　the　sequence.　Regions　having　significant　outstanding　appearance　are　regarded　as　key　regions.　Binarization　and　greedy　algorithm　are　used　to　choose　key　regions　and　discriminative　patterns　with　low　correlation.　The　key　frames　are　key　checking　positions　of　the　video　path,　whilst　all　other　frames　are　tracked　by　matching　approaches　with　substantially　reduced　computation.　In　the　online　part,　each　live　frame　is　used　initially　to　find　the　most　nearby　key　frame,　and　the　computation　power　of　the　subsequent　detection　is　substantially　reduced　by　looking　for　the　next　key　frame　with　the　frame　by　frame　tracking　procedure.　Practical　field　tests　were　done　on　real　data　of　the　light　railway　system　in　Hong　Kong.　Results　of　these　experimental　tests　show　that　the　approach　can　identify　almost　100%　pre-recorded　scene　along　railway　paths　with　pedestrians.　The　approach　has　shown　better　performance　over　conventional　approaches　using　some　standard　video　sequences　for　scene　recognition.　©　2018　IEEE.