The　signal-to-noise　ratio　and　resolution　of　the　reconstructed　image　is　seriously　influenced　by　speckle　noise　in　digital　holography,　so　it　is　essential　to　reduce　the　speckle　noise　and　improve　the　image　quality.　Intensity　correlation　between　two　speckle　patterns　with　the　rectangle　speckle　spot　is　analyzed　and　deduced,　and　the　minimal　angular　difference　of　illumination　beams　is　given　quantitatively　under　a　special　situation.　A　lensless　Fourier　transform　holographic　imaging　system　with　angular　diversity　is　designed　based　on　the　lens　property,　in　which　the　direction　of　the　wave　can　be　changed　by　shifting　the　fiber　instead　of　conventionally　rotating　the　mirror,　and　the　formor　can　change　the　direction　of　illumination　with　a　fixed　illumination　size　and　location.　Eighty　one　holograms　with　uncorrelated　speckle　patterns　are　recorded　at　different　illumination　angles　by　shifting　the　fiber.　Then　a　digital　reconstruction　is　achieved　only　by　a　fast　Fourier　transform,　and　the　multiple　reconstructed　images　are　averaged.　Experimental　results　show　that　the　speckle　contrast　of　the　averaged　reconstructed　image　can　be　reduced　to　14.6%　that　from　a　single　reconstructed　image,　and　the　signal-to-noise　ratio　is　improved　6.9　times.　This　proposed　digital　holographic　imaging　method　can　suppress　the　speckle　noise　greatly,　has　a　simple　setup,　and　is　easy　to　operate.