In　order　to　model　high-speed　nonlinear　circuits,　recurrent　neural　network　(RNN)　has　been　widely　used　in　computer-aided　design　(CAD)　area　to　achieve　high　performance　and　fast　models　compared　with　the　existing　models.　Despite　their　advantages,　they　still　have　challenges　such　as　large　training　time　and　limited　test　accuracy.　In　this　article,　the　batch　normalization　(BN)　method　is　applied　to　deep　RNN　leading　to　a　much　shorter　training　time　and　more　accurate　models　compared　with　the　conventional　RNN.　The　proposed　BN-RNN　method　works　by　modifying　the　distribution　of　the　internal　nodes　of　a　deep　network　in　the　training　course　as　an　internal　auxiliary　shift　yielding　a　much　faster　training.　Indeed,　the　internal　covariance　shift　will　be　reduced　and　the　training　of　deep　neural　networks　will　be　accelerated　via　a　normalization　step　applied　to　the　layers　of　RNN.　BN-RNN,　moreover,　has　a　beneficial　effect　on　gradient　flow　through　the　grid　by　reducing　the　dependence　of　gradients　on　the　scale　of　network　parameters　or　their　initial　values.　This　provides　a　much　better　learning　process　without　the　risk　of　divergence.　For　verifying　the　proposed　method,　time-domain　modeling　of　three　high-speed　nonlinear　circuits　operating　at　the　GHz　region　is　provided.　Comparisons　of　the　training　and　test　errors　between　RNN　and　BN-RNN,　and　evaluation　time　comparisons　between　transistor　level　and　the　BN-RNN-based　models　for　these　circuits　prove　the　higher　speed　of　the　models　obtained　from　the　BN-RNN　method.　In　addition,　it　is　shown　that　training　using　the　proposed　method　requires　much　less　CPU　time　and　number　of　epochs.