This　article　introduces　the　deep　neural　network　method　into　the　field　of　high-dimensional　microwave　modeling.　Deep　learning　is　nowadays　highly　successful　in　solving　complex　and　challenging　pattern　recognition　and　classification　problems.　This　article　investigates　the　use　of　deep　neural　networks　to　solve　microwave　modeling　problems　that　are　much　more　challenging　than　that　solved　by　the　previous　shallow　neural　networks.　The　most　commonly　used　activation　function　in　the　existing　deep　neural　network　is　the　rectified　linear　unit　(ReLU),　which　is　a　piecewise　hard　switch　function.　However,　such　a　ReLU　is　not　suitable　for　microwave　modeling　where　the　inputoutput　relationships　are　smooth　and　continuous.　In　this　article,　we　propose　a　new　deep　neural　network　to　perform　high-dimensional　microwave　modeling.　A　smooth　ReLU　is　proposed　for　the　new　deep　neural　network.　The　proposed　deep　neural　network　employs　both　the　sigmoid　function　and　the　smooth　ReLU　as　activation　functions.　The　new　deep　neural　network　can　represent　the　smooth　inputoutput　relationship　that　is　required　for　microwave　modeling.　An　advanced　three-stage　deep　learning　algorithm　is　proposed　to　train　the　new　deep　neural　network　model.　This　algorithm　can　determine　the　number　of　hidden　layers　with　sigmoid　functions　and　those　with　smooth　ReLUs　in　the　training　process.　It　can　also　overcome　the　vanishing　gradient　problem　for　training　the　deep　neural　network.　The　proposed　deep　neural　network　technique　can　solve　microwave　modeling　problems　in　a　higher　dimension　than　the　previous　neural　network　method,　i.e.,　shallow　neural　network　method.　Two　high-dimensional　parameter-extraction　modeling　examples　of　microwave　filters　are　presented　to　demonstrate　the　proposed　deep　neural　network　technique.