In　most　echo　state　networks　(ESNs),　the　training　error　typically　decreases　as　the　network　size　increases,　and　thus　the　overfitting　issue　is　widely　existed.　To　solve　this　problem,　an　incremental　ESN　(IESN)　is　proposed　by　incorporating　the　leave-one-out　cross-validation　(LOO-CV)　and　the　regularization　method.　First,　the　LOO-CV　error　is　used　to　automatically　identify　the　network　architecture　such　that　the　overfitting　problem　is　avoided　to　some　extent.　Second,　the　regularization　technique　is　used　to　solve　the　ill-posed　problem,　and　thus　the　IESN　owns　good　robustness　property.　Third,　the　output　weights　are　incrementally　calculated　by　the　fast　SVD　updating　algorithm　to　reduce　the　ESN　training　time.　Moreover,　the　stability　and　convergence　of　IESN　are　discussed　to　ensure　its　successful　application.　Simulation　results　demonstrate　that　the　proposed　IESN　requires　fewer　reservoir　nodes　yet　obtains　much　better　performance　than　other　existing　ESNs.