Traditional　model-based　tracking　methods　apply　template　of　changeless　size　and　shape　to　represent　the　object,　which　are　only　suitable　for　target　translation.　Whereas,　in　the　course　of　tracking　there　are　various　transformations　for　the　object,　including　flip,　rotation,　scaling　and　shear.　This　paper　presents　a　new　method　that　can　perform　well　in　coping　with　object＇s　affine　transformation　and　make　the　template　follow　the　object＇s　state　duly　and　precisely.　In　the　related　algorithm,　Newton-Downhill　method　is　utilized　to　calculate　the　six　parameters　of　affine　transform　model.　The　template　is　updated　based　on　the　final　matching　result,　modulating　its　size　and　shaping　to　follow　the　temporal　evolution　of　the　object.　©2010　IEEE.