Cross-Validation　(CV)　is　the　primary　mechanism　used　in　Machine　Learning　to　control　generalization　error　in　the　absence　of　sufficiently　large　quantities　of　marked　up　(tagged　or　labelled)　data　to　undertake　independent　testing,　training　and　validation　(including　early　stopping,　feature　selection,　parameter　tuning,　boosting　and/or　fusion).　Repeated　Cross-Validation　(RCV)　is　used　to　try　to　further　improve　the　accuracy　of　our　performance　estimates,　including　compensating　for　outliers.　Typically　a　Machine　Learning　researcher　will　the　compare　a　new　target　algorithm　against　a　wide　range　of　competing　algorithms　on　a　wide　range　of　standard　datasets.　The　combination　of　many　training　folds,　many　CV　repetitions,　many　algorithms　and　parameterizations,　and　many　training　sets,　adds　up　to　a　very　large　number　of　data　points　to　compare,　and　a　massive　multiple　testing　problem　quadratic　in　the　number　of　individual　test　combinations.　Research　in　Machine　Learning　sometimes　involves　basic　significance　testing,　or　provides　confidence　intervals,　but　seldom　addresses　the　multiple　testing　problem　whereby　the　assumption　of　p