The　fine-grained　ship　image　recognition　task　aims　to　identify　various　classes　of　ships.　However,　small　inter-class,　large　intra-class　differences　between　ships,　and　lacking　of　training　samples　are　the　reasons　that　make　the　task　difficult.　Therefore,　to　enhance　the　accuracy　of　the　fine-grained　ship　image　recognition,　we　design　a　fine-grained　ship　image　recognition　network　based　on　bilinear　convolutional　neural　network　(BCNN)　with　Inception　and　additive　margin　Softmax　(AM-Softmax).　This　network　improves　the　BCNN　in　two　aspects.　Firstly,　by　introducing　Inception　branches　to　the　BCNN　network,　it　is　helpful　to　enhance　the　ability　of　extracting　comprehensive　features　from　ships.　Secondly,　by　adding　margin　values　to　the　decision　boundary,　the　AM-Softmax　function　can　better　extend　the　inter-class　differences　and　reduce　the　intra-class　differences.　In　addition,　as　there　are　few　publicly　available　datasets　for　fine-grained　ship　image　recognition,　we　construct　a　Ship-43　dataset　containing　47,300　ship　images　belonging　to　43　categories.　Experimental　results　on　the　constructed　Ship-43　dataset　demonstrate　that　our　method　can　effectively　improve　the　accuracy　of　ship　image　recognition,　which　is　4.08%　higher　than　the　BCNN　model.　Moreover,　comparison　results　on　the　other　three　public　fine-grained　datasets　(Cub,　Cars,　and　Aircraft)　further　validate　the　effectiveness　of　the　proposed　method.