In　order　to　improve　the　performance　of　the　tuned　mass　damper　(TMD)　for　machining　chatter　suppression,　a　new-type　of　nonlinear　TMD　is　proposed　in　this　paper.　Compared　with　the　common　linear　TMD,　the　nonlinear　TMD　is　equipped　with　an　additional　series　friction-spring　element.　The　capability　of　the　nonlinear　TMD　in　suppressing　machining　chatter　vibration　is　investigated　in　this　paper.　The　harmonic　balancing　method　(HBM)　is　used　to　estimate　the　frequency　response　function　(FRF)　of　the　machining　system　to　which　the　nonlinear　TMD　is　attached.　Considering　the　special　nature　of　the　machining　stability　problem,　the　optimal　design　parameters　of　this　nonlinear　TMD　are　those　that　minimize　the　magnitude　of　the　real　part　of　the　FRF　of　the　nonlinear　TMD　damped　machining　system.　This　paper　also　demonstrates　the　performance　of　the　optimally　tuned　nonlinear　TMD　for　machining　stability　improvement　by　calculating　the　stability　diagrams　for　the　milling　of　the　nonlinear　TMD　damped　workpiece.　The　calculation　results　show　that　more　than　30%　improvement　in　the　critical　limiting　cutting　depth　can　be　obtained,　compared　to　the　optimally　tuned　linear　TMD.　(C)　2010　Elsevier　Ltd.　All　rights　reserved.