Privacy-preserving　empirical　risk　minimization　model　is　crucial　for　the　increasingly　frequent　setting　of　analyzing　personal　data,　such　as　medical　records,　financial　records,　etc.　Due　to　its　advantage　of　a　rigorous　mathematical　definition,　differential　privacy　has　been　widely　used　in　privacy　protection　and　has　received　much　attention　in　recent　years　of　privacy　protection.　With　the　advantages　of　iterative　algorithms　in　solving　a　variety　of　problems,　like　empirical　risk　minimization,　there　have　been　various　works　in　the　literature　that　target　differentially　private　iteration　algorithms,　especially　the　adaptive　iterative　algorithm.　However,　the　solution　of　the　final　model　parameters　is　imprecise　because　of　the　vast　privacy　budget　spending　on　the　step　size　search.　In　this　paper,　we　first　proposed　a　novel　adaptive　differential　privacy　algorithm　that　does　not　require　the　privacy　budget　for　step　size　determination.　Then,　through　the　theoretical　analyses,　we　prove　that　our　proposed　algorithm　satisfies　differential　privacy,　and　their　solutions　achieve　sufficient　accuracy　by　infinite　steps.　Furthermore,　numerical　analysis　is　performed　based　on　real-world　databases.　The　results　indicate　that　our　proposed　algorithm　outperforms　existing　algorithms　for　model　fitting　in　terms　of　accuracy.