The　visual-based　scene　recognition　and　localization　module　is　widely　used　in　vehicle　safety　system.　This　paper　proposes　a　new　method　of　scene　recognition　based　on　local　key　region　and　key　frame,　which　is　based　on　the　problem　of　large　amount　of　training　data,　large　matching　complexity　and　low　tracking　precision.　The　proposed　method　meets　the　real-time　requirements　with　high　accuracy.　First,　the　method　uses　the　unsupervised　method　to　extract　the　significant　regions　of　the　single　reference　sequence　captured　by　the　monocular　camera　as　the　key　regions.　The　binary　features　with　low　correlation　in　key　regions　are　also　extracted　to　improve　the　scene　matching　accuracy　and　reduce　the　computational　complexity　of　feature　generation　and　matching.　Secondly,　key　frames　in　the　reference　sequence　are　extracted　based　on　the　discrimination　score　to　reduce　the　retrieval　range　of　the　tracking　module　and　improve　the　efficiency.　Practical　field　tests　are　done　on　real　data　of　the　light　railway　system　in　Hong　Kong　and　the　open　test　data　set　in　Nordland.　The　experimental　results　show　that　the　proposed　method　achieves　fast　matching　and　the　precision　is　9.8%　higher　than　SeqSLAM　which　is　based　on　global　feature.　©　2018,　Science　Press.　All　right　reserved.