Statistical comparison of classifiers through Bayesian hierarchical modelling

Corani, Giorgio and Benavoli, Alessio and Demšar, Janez and Mangili, Francesca and Zaffalon, Marco (2016) Statistical comparison of classifiers through Bayesian hierarchical modelling. Technical Report. UNSPECIFIED.

[img] Text
hierarchical.pdf

Download (420kB)

Abstract

We propose a new approach for the statistical comparison of algorithms which have been cross-validated on multiple data sets. It is a Bayesian hierarchical method; it draws inferences on single and on multiple datasets taking into account the mean and the variability of the cross-validation results. It is able to detect equivalent classi ers and to claim signi cances which have a practical impact. On each data sets it estimates more accurately than the existing methods the di erence of accuracy between the two classi ers thanks to shrinkage. Such advantages are demonstrated by simulations on synthetic and real data.

Actions (login required)

View Item View Item