In　this　paper,　the　bit-sensitivity　of　H.264　bitstream　has　been　studied　firstly,　to　detect　the　most　significant　bits　for　video　reconstruction.　And　then,　an　efficient　video　encryption　scheme　is　presented.　In　the　proposed　scheme,　the　most　significant　bits　in　H.264　bitstream　are　directly　extracted　to　be　encrypted,　including　the　codewords　of　intra-prediction　modes,　motion　vector　difference　and　the　sign　bits　of　the　low　frequency　coefficients　of　intra-frames.　Owing　to　the　proposed　scheme　is　independent　of　the　compression　process,　thus　does　not　need　to　modify　the　structure　of　the　H.264　standard　codec.　Experimental　results　show　that　the　proposed　scheme　exhibits　reliable　perceptual　security,　can　secure　against　not　only　replacement　attacks　but　also　brute-force　attacks,　and　achieves　significant　computational　efficiency.　Furthermore,　it　maintains　the　format-compliance　to　the　H.264　standard　decoder　and　has　no　escalation　in　bitrates.