To　improve　synthesis　realistic　of　sign　language　videos,　a　method　to　describe　sign　language　video　semantics　is　proposed,　and　the　sign　language　video　database　based　on　semantic　description　for　sign　language　synthesis　is　constructed.　Chinese　sign　language　videos　in　specific　research　field　are　captured,　then　sign　language　video　units　and　multi-dimensional　transition　units　are　cut　from　the　captured　sign　language　videos.　By　describing　the　semantic　information　of　every　frame　in　sign　language　videos,　which　include　locations,　hand　shapes　and　rhythm　information,　their　multi-dimensional　semantic　models　are　constructed.　During　sign　language　video　synthesis,　useful　information　can　be　used　in　real-time　by　parsing　multi-dimensional　semantic　models.　This　method　provides　real-time　and　coherent　semantic　information　for　sign　language　video　synthesis,　and　in　the　process　of　joining　two　sign　language　videos,　different　rhythm　information　can　be　parsed　out　from　their　semantic　models,　then　interpolated　locations　of　transition　frames　can　be　moderately　adjusted　to　make　the　rhythm　in　transition　frames　gradually　change.