Objects　in　unmanned　aerial　vehicle　(UAV)　images　are　generally　small　due　to　the　high-photography　altitude.　Although　many　efforts　have　been　made　in　object　detection,　how　to　accurately　and　quickly　detect　small　objects　is　still　one　of　the　remaining　open　challenges.　In　this　paper,　we　propose　a　feature　fusion　and　scaling-based　single　shot　detector　(FS-SSD)　for　small　object　detection　in　the　UAV　images.　The　FS-SSD　is　an　enhancement　based　on　FSSD,　a　variety　of　the　original　single　shot　multibox　detector　(SSD).　We　add　an　extra　scaling　branch　of　the　deconvolution　module　with　an　average　pooling　operation　to　form　a　feature　pyramid.　The　original　feature　fusion　branch　is　adjusted　to　be　better　suited　to　the　small　object　detection　task.　The　two　feature　pyramids　generated　by　the　deconvolution　module　and　feature　fusion　module　are　utilized　to　make　predictions　together.　In　addition　to　the　deep　features　learned　by　the　FS-SSD,　to　further　improve　the　detection　accuracy,　spatial　context　analysis　is　proposed　to　incorporate　the　object　spatial　relationships　into　object　redetection.　The　interclass　and　intraclass　distances　between　different　object　instances　are　computed　as　a　spatial　context,　which　proves　effective　for　multiclass　small　object　detection.　Six　experiments　are　conducted　on　the　PASCAL　VOC　dataset　and　the　two　UAV　image　datasets.　The　experimental　results　demonstrate　that　the　proposed　method　can　achieve　a　comparable　detection　speed　but　an　accuracy　superior　to　those　of　the　six　state-of-the-art　methods.