The　development　of　intelligent　transportation　systems　(ITS)　has　attracted　significant　attention　to　connected　and　autonomous　vehicles　(CAVs).　It　is　urgent　to　investigate　multi-CAV　intelligent　cruise　control　solutions　in　mixed　traffic　environments.　In　addition,　the　impact　of　platoon　dynamics　and　time　delays,　induced　by　shared　wireless　communications,　data　processing,　and　actuation　cannot　be　ignored.　This　article　investigates　the　development　of　a　multi-agent　deep　reinforcement　learning　(MADRL)　controller　tailored　for　CAVs　operating　within　mixed　and　dynamic　traffic　scenarios　that　involve　time　delays.　Firstly,　the　error　dynamics　in　the　discrete-time　domain　for　each　subplatoon　is　derived　by　considering　the　time-varying　delays　and　leading　vehicle　states,　and　then　the　optimal　CAV　cruise　control　problem　is　formulated.　Subsequently,　the　partially　observable　Markov　game　(POMG)　is　used　to　construct　the　multi-agent　environment,　and　then　a　centralized　training　decentralized　execution　(CTDE)　algorithm　framework　is　proposed　based　on　the　multi-agent　deep　deterministic　policy　gradient　(MADDPG)　method.　Finally,　the　computational　complexity　and　the　influence　of　delay　are　analyzed.　The　simulation　results　illustrate　the　effectiveness　of　the　proposed　intelligent　algorithm.　©　2024　IEEE.