Echo　state　networks　(ESNs)　have　been　widely　used　in　the　field　of　time　series　prediction.　However,　it　is　difficult　to　automatically　determine　the　structure　of　ESN　for　a　given　task.　To　solve　this　problem,　the　dynamical　regularized　ESN　(DRESN)　is　proposed.　Different　from　other　growing　ESNs　whose　existing　architectures　are　fixed　when　new　reservoir　nodes　are　added,　the　current　component　of　DRESN　may　be　replaced　by　the　newly　generated　network　with　more　compact　structure　and　better　prediction　performance.　Moreover,　the　values　of　output　weights　in　DRESN　are　updated　by　the　error　minimization-based　method,　and　the　norms　of　output　weights　are　controlled　by　the　regularization　technique　to　prevent　the　ill-posed　problem.　Furthermore,　the　convergence　analysis　of　the　DRESN　is　given　theoretically　and　experimentally.　Simulation　results　demonstrate　that　the　proposed　approach　can　have　few　reservoir　nodes　and　better　prediction　accuracy　than　other　existing　ESN　models.