Aiming　at　the　problem　that　the　numbers　of　operant　actions　are　more　in　operant　conditioning　probabilistic　automaton,　this　paper　constructs　a　hierarchical　structural　operant　conditioning　probabilistic　automaton,　which　is　called　as　HS-OCPA　bionic　autonomous　learning　system.　The　HS-OCPA　learning　system　which　doesn＇t　require　the　system　model　is　designed　mainly　based　on　Skinner　operant　conditioning　(Skinner　OC)　mechanism　and　probabilistic　automata　(PA).　The　HS-OCPA　learning　system　uses　OC　learning　mechanism　to　realize　optimizing　learning　based　on　operant　actions　and　system　performance,　and　the　OC　learning　mechanism　is　adjusted　by　the　reorientation　information　of　operant　actions.　Finally　the　optimal　control　strategy　is　searched　on　line.　The　simulation　and　experiment　applied　in　two-wheeled　robot　poster　balance　control　both　show　that　the　designed　HS-OCPA　learning　system　not　only　has　quickly　learning　velocity　but　also　has　strong　adaptive　ability　when　numbers　of　operant　actions　are　more.