In　the　cooperation　between　humans　and　robots,　it　is　essential　for　robots　to　translate　natural　human　language　into　continuous　action　sequences　to　complete　complex　collaborative　tasks.　In　this　article,　a　hierarchical　system　is　established　to　realize　the　conversion　from　human　natural　language　to　robot　motion　sequence　in　complex　tasks.　The　system　consists　of　three　layers:　task　layer　(top　layer),　semantic　motion　layer　(middle　layer)　and　motion　primitive　layer　(bottom　layer).　When　humans　tell　robots　tasks　through　natural　language,　they　first　input　language　sentences　at　the　task　level;　Then,　by　combining　oral　description　with　visual　cues,　the　sentence　is　translated　into　motion　language　in　the　middle　layer,　which　consists　of　the　predicate　object　structure　and　the　six-dimensional　state　(position　and　posture)　of　the　object.　Among　them,　the　sequence　of　predicate　object　structure　uses　words　to　describe　complex　tasks　at　semantic　level,　and　the　6D　state　of　the　object　mainly　includes　the　initial　state　before　operation　and　the　target　state　after　operation.　In　addition,　we　propose　a　novel　search　algorithm　of　motion　sequences　which　integrates　our　knowledge　base　with　Deep　Q-learning.　Furthermore,　the　new　knowledge　base　is　established　which　is　used　to　encode　various　characteristics　of　motions,　objects　and　relationships.　To　verify　the　effectiveness　of　this　method,　we　set　up　an　actual　robot　experimental　platform　(consisting　of　aubo-i5　manipulator　and　robotiq　mechanical　claw)　for　typical　complex　operation　experiments.　©　2023,　The　Author(s),　under　exclusive　license　to　Springer　Nature　Singapore　Pte　Ltd.