Reinforcement　learning　is　a　powerful　method　for　solving　sequential　decision　making　problems.　But　it　is　difficult　to　apply　to　practical　problems　such　as　multi-agent　systems　with　continuous　state　space　problems.　In　this　paper　we　present　a　cooperative　strategy　learning　method　to　solve　the　problem.　It　combines　WOLF-PHC　algorithms　with　function　approximation　of　RL　techniques.　By　this　method　an　agent　could　learn　cooperative　behavior　in　the　multi-agent　environment　with　continuous　state　space.　Using　a　subtask　of　RoboCup　soccer,　Keepaway,　we　demonstrate　the　effective　of　this　learning　method　and　the　experiment　results　show　that　the　algorithm　converges.