Purpose　The　purpose　of　this　paper　is　to　propose　a　stable　high-order　absorbing　boundary　condition　(ABC)　based　on　new　continued　fraction　for　scalar　wave　propagation　in　2D　and　3D　unbounded　layers.　Design/methodology/approach　The　ABC　is　obtained　based　on　continued　fraction　(CF)　expansion　of　the　frequency-domain　dynamic　stiffness　coefficient　(DtN　kernel)　on　the　artificial　boundary　of　a　truncated　infinite　domain.　The　CF　which　has　been　used　to　the　thin　layer　method　in　[69]　will　be　applied　to　the　DtN　method　to　develop　a　time-domain　high-order　ABC　for　the　transient　scalar　wave　propagation　in　2D.　Furthermore,　a　new　stable　composite-CF　is　proposed　in　this　study　for　3D　unbounded　layers　by　nesting　the　above　CF　for　2D　layer　and　another　CF.　Findings　The　ABS　has　been　transformed　from　frequency　to　time　domain　by　using　the　auxiliary　variable　technique.　The　high-order　time-domain　ABC　can　couple　seamlessly　with　the　finite　element　method.　The　instability　of　the　ABC-FEM　coupled　system　is　discussed　and　cured.　Originality/value　This　manuscript　establishes　a　stable　high-order　time-domain　ABC　for　the　scalar　wave　equation　in　2D　and　3D　unbounded　layers,　which　is　based　on　the　new　continued　fraction.　The　high-order　time-domain　ABC　can　couple　seamlessly　with　the　finite　element　method.　The　instability　of　the　coupled　system　is　discussed　and　cured.