The　wastewater　treatment　process　(WWTP)　is　beneficial　for　maintaining　sufficient　water　resources　and　recycling　wastewater.　A　crucial　link　of　WWTP　is　to　ensure　that　the　dissolved　oxygen　(DO)　concentration　is　continuously　maintained　at　the　predetermined　value,　which　can　actually　be　considered　as　a　tracking　problem.　In　this　article,　an　experience　replay-based　action-dependent　heuristic　dynamic　programming　(ER-ADHDP)　method　is　developed　to　design　the　model-free　tracking　controller　to　accomplish　the　tracking　goal　of　the　DO　concentration.　First,　the　online　ER-ADHDP　controller　is　regarded　as　a　supplementary　controller　to　conduct　the　model-free　tracking　control　alongside　a　stabilizing　controller　with　a　priori　knowledge.　The　online　ER-ADHDP　method　can　adaptively　adjust　weight　parameters　of　critic　and　action　networks,　thereby　continuously　ameliorating　the　tracking　result　over　time.　Second,　the　ER　technique　is　integrated　into　the　critic　and　action　networks　to　promote　the　data　utilization　efficiency　and　accelerate　the　learning　process.　Third,　a　rational　stability　result　is　provided　to　theoretically　ensure　the　usefulness　of　the　ER-ADHDP　tracking　design.　Finally,　simulation　experiments　including　different　reference　trajectories　are　conducted　to　show　the　superb　tracking　performance　and　excellent　adaptability　of　the　proposed　ER-ADHDP　method.