This　article　considers　an　energy-harvesting　wireless　sensor　network　(EH-WSN)　where　nodes　have　a　dual　alternative　battery　system.　We　aim　to　design　a　routing　protocol　that　transmits　a　given　amount　of　data　to　a　sink　using　the　minimum　number　of　time　slots.　To　this　end,　we　first　formulate　the　routing　problem　as　a　mixed-integer　linear　program　(MILP)　and　model　the　charging　and　discharging　process　at　each　node.　Moreover,　we　also　propose　a　distributed　multiarmed　bandit-based　routing　algorithm　(MABRA),　where　each　node　learns　the　best　next　hop(s)　to　forward　its　data　based　on　the　battery　and　data　information　of　its　neighbors.　We　study　how　the　number　of　nodes　that　generate　data　or　sources,　solar　panel　area,　and　path　exploration　probability　impacts　the　end-toend　delays　computed　by　MILP　and　MABRA.　Numerical　results　show　that　the　end-to-end　delays　computed　by　MILP　increase　with　the　number　of　sources.　It,　however,　decreases　if　nodes　use　larger　solar　panels.　Critically,　nodes　experience　end-to-end　delays　that　are　55%　lower　than　the　average　end-to-end　delay　of　a　competing　routing　algorithm.