With　the　deepening　of　modernization　and　industrialization,　the　issues　of　water　pollution　and　scarcity　have　become　more　pressing.　To　address　these　issues,　many　wastewater　treatment　factories　have　been　built　to　improve　the　reuse　of　water　resources.　However,　the　control　of　the　wastewater　treatment　process　(WWTP)　is　a　complex　task　due　to　the　highly　nonlinear　and　strongly　coupled　nature.　It　is　challenging　to　develop　the　accurate　mechanism　models　of　the　wastewater　treatment　system.　The　improvement　of　the　efficiency　for　the　WWTP　is　crucial　to　safeguard　the　urban　ecological　environment.　In　this　paper,　adaptive　critic　with　weight　allocation　(ACWA)　is　developed　to　address　the　optimal　control　problem　in　the　WWTP.　Different　from　the　previous　methods　of　the　WWTP,　system　modeling　is　not　adopted　in　this　paper,　which　meets　the　actual　physical　background　of　the　wastewater　treatment　system　to　a　great　extent.　In　addition,　the　actor　-critic　algorithm　in　reinforcement　learning　is　used　as　the　basic　structure　in　the　ACWA.　It　is　worth　noting　that　a　novel　weighted　action　-value　function　and　the　advantage　function　are　introduced　in　the　weight　updating　process　of　the　action　network　and　the　critic　network.　The　experimental　results　show　that　the　control　accuracy　of　the　ACWA　is　greatly　improved　compared　with　the　previous　control　methods.