Sparse　deep　neural　networks　(SDNNs)　for　speaker　segmentation　are　proposed.　First,　the　SDNNs　are　trained　using　the　side　information　that　is　the　class　label　of　the　input.　Then,　speaker-specific　features　are　extracted　from　the　super-vector　feature　of　the　speech　signal　by　the　SDNNs.　Lastly,　the　label　of　each　speech　frame　is　obtained　by　K-means　clustering,　which　is　used　to　segment　different　speakers　of　a　continuous　speech　stream.　The　performance　evaluation　using　the　multi-speaker　speech　stream　corpus　generated　from　the　TIMIT　database　shows　that　the　proposed　speaker　segmentation　algorithm　outperforms　the　Bayesian　information　criterion　method　and　the　deep　auto-encoder　networks　method.