In　device-to-device　(D2D)　communications,　the　channel　gain　between　a　transmitter　and　a　receiver　is　difficult　to　predict　due　to　channel　variations.　Hence,　an　attacker　can　easily　perform　an　impersonation　attack　between　two　authentic　D2D　users.　As　a　countermeasure,　we　propose　a　reinforcement　learning-based　technique　that　guarantees　identification　of　the　impersonator　based　on　channel　gains.　To　show　the　merit　of　our　technique,　we　report　its　performance　in　terms　of　false　alarm　rate,　miss-detection　rate,　and　average　error　rate.　The　secret　key　generation　rate　is　also　determined　under　the　impersonation　attack　based　on　physical　layer　security.