Action　recognition　in　realistic　scenes　is　a　challenging　task　in　the　field　of　computer　vision.　Although　trajectory-based　methods　have　demonstrated　promising　performance,　background　trajectories　cannot　be　filtered　out　effectively,　which　leads　to　a　reduction　in　the　ratio　of　valid　trajectories.　To　address　this　issue,　we　propose　a　saliency-based　sampling　strategy　named　foreground　trajectories　on　multiscale　hybrid　masks　(HM-FTs).　First,　the　motion　boundary　images　of　each　frame　are　calculated　to　derive　the　initial　masks.　According　to　the　characteristics　of　action　videos,　image　priors　and　the　synchronous　updating　mechanism　based　on　cellular　automata　are　exploited　to　generate　an　optimized　weak　saliency　map,　which　will　be　integrated　with　a　strong　saliency　map　obtained　via　the　multiple　kernels　boosting　algorithm.　Then,　multiscale　hybrid　masks　are　achieved　through　the　collaborative　optimization　strategy　and　masks　intersection.　The　compensation　schemes　are　designed　to　extract　a　set　of　foreground　trajectories　that　are　closely　related　to　human　actions.　Finally,　a　hybrid　fusion　framework　for　combining　trajectory　features　and　pose　features　is　constructed　to　enhance　the　recognition　performance.　The　experimental　results　on　two　benchmark　datasets　demonstrate　that　the　proposed　method　is　effective　and　improves　upon　most　of　the　state-of-the-art　algorithms.　(c)　The　Authors.　Published　by　SPIE　under　a　Creative　Commons　Attribution　3.0　Unported　License.　Distribution　or　reproduction　of　this　work　in　whole　or　in　part　requires　full　attribution　of　the　original　publication,　including　its　DOI.