Colorectal　cancer(CRC)　is　a　significant　health　problem　in　the　world,　the　incidence　of　CRC　can　be　largely　preventable　by　early　detection　and　removal　of　the　polyps　before　they　turn　into　the　malignant　structure.　Most　existing　CAD　system　for　polyps　detection　rely　on　fully　supervised　learning　which　requires　the　tedious　manual　annotation　and　precise　colon　segmentation.　This　paper　proposed　a　method　based　on　multiple　instance　learning　and　transfer　learning.　Our　scheme　firstly　extracts　many　small　patches　from　CTC　images　by　using　threshold　segmentation　method,　then　a　pre-trained　model　was　applied　for　feature　extracting　of　instances,　next　pooling　operator　was　used　to　aggregating　these　instance　features　into　a　bag,　finally,　classification　result　was　obtained　by　a　classifier.　Our　proposed　method　does　not　rely　on　accurate　colon　segmentation　and　the　result　show　that　it　can　achieve　a　high　accuracy　rate.　Copyright　©　2020　by　SCITEPRESS　–　Science　and　Technology　Publications,　Lda.　All　rights　reserved