Stress　has　been　identified　as　one　of　major　contributing　factors　in　car　crashes　due　to　its　negative　impact　on　driving　performance.　It　is　in　urgent　need　that　the　stress　levels　of　drivers　can　be　detected　in　real　time　with　high　accuracy　so　that　intervening　or　navigating　measures　can　be　taken　in　time　to　mitigate　the　situation.　Existing　driver　stress　detection　models　mainly　rely　on　traditional　machine　learning　techniques　to　fuse　multimodal　data.　However,　due　to　the　non-linear　correlations　among　modalities,　it　is　still　challenging　for　traditional　multimodal　fusion　methods　to　handle　the　real-time　influx　of　complex　multimodal　and　high　dimensional　data,　and　report　drivers?　stress　levels　accurately.　To　solve　this　issue,　a　framework　of　driver　stress　detection　through　multimodal　fusion　using　attention　based　deep　learning　techniques　is　proposed　in　this　paper.　Specifically,　an　attention　based　convolutional　neural　networks　(CNN)　and　long　short-term　memory　(LSTM)　model　is　proposed　to　fuse　non-invasive　data,　including　eye　data,　vehicle　data,　and　environmental　data.　Then,　the　proposed　model　can　automatically　extract　features　separately　from　each　modality　and　give　different　levels　of　attention　to　features　from　different　modalities　through　self-attention　mechanism.　To　verify　the　validity　of　the　proposed　method,　extensive　experiments　have　been　carried　out　on　our　dataset　collected　using　an　advanced　driving　simulator.　Experimental　results　demonstrate　that　the　performance　of　the　proposed　method　on　driver　stress　detection　outperforms　the　state-of-the-art　models　with　an　average　accuracy　of　95.5%.