Deep　learning　architectures　which　are　exceptionally　deep　have　exhibited　to　be　incredibly　powerful　models　for　image　processing.　As　the　architectures　become　deep,　it　introduces　challenges　and　difficulties　in　the　training　process　such　as　overfitting,　computational　cost,　and　exploding/vanishing　gradients　and　degradation.　A　new　state-of-the-art　densely　connected　architecture,　called　DenseNets,　has　exhibited　an　exceptionally　outstanding　result　for　image　classification.　However,　it　still　computationally　costly　to　train　DenseNets.　The　choice　of　the　activation　function　is　also　an　important　aspect　in　training　of　deep　learning　networks　because　it　has　a　considerable　impact　on　the　training　and　performance　of　a　network　model.　Therefore,　an　empirical　analysis　of　some　of　the　nonlinear　activation　functions　used　in　deep　learning　is　done　for　image　classification.　The　activation　functions　evaluated　include　ReLU,　Leaky　ReLU,　ELU,　SELU　and　an　ensemble　of　SELU　and　ELU.　Publicly　available　datasets　Cifar-10,　SVHN,　and　PlantVillage　are　used　for　evaluation.　Copyright　©　2020　Inderscience　Enterprises　Ltd.