Surveillance　video　based　abnormal　crowd　event　recognition　has　important　practical　significance　in　public　safety　management.　Traditional　methods　usually　based　on　modeling　and　analysis　of　certain　feature　modes,　and　the　performances　are　quite　sensitive　to　the　features　adopted.　However,　there　are　many　kinds　of　event　to　be　recognized　and　many　features　can　be　used,　how　to　find　the　most　effective　features　to　distinguish　a　certain　kind　of　event　is　quite　a　difficult　problem　to　solve.　Convolutional　Neural　Network　has　the　characteristics　of　simple　structure,　less　training　parameters　and　strong　adaptability.　It　can　learn　effective　features　and　then　establish　corresponding　models　automatically　from　a　large　number　of　training　data.　In　this　paper,　a　crowd　event　recognition　algorithm　was　proposed　based　on　inter-frame　differences　and　convolutional　neural　network.　Where　the　inter-frame　difference　is　used　to　extract　the　most　original　motion　characteristics　of　the　crowd　video,　then　to　be　selected　by　a　LeNet-5　based　network　to　get　the　most　effective　ones.　Where　the　LeNet-5　model　was　improved　redesigned　for　crowd　video　analysis　and　events　recognition.　The　performance　of　the　proposed　algorithm　was　evaluated　by　dataset　PETS　2009　The　results　show　that　the　recognition　accuracy　of　the　proposed　algorithm　reaches　over　96%,　which　is　much　higher　than　the　traditional　feature　model　based　methods,　as　well　as　better　robustness　in　different　views.