From: Stefano Ciulli <stefano.ciulli@gmail.com>
Reply-To: "Weka machine learning workbench list." <wekalist@list.scms.waikato.ac.nz>
Date: Wednesday, 24 April 2013 12:21 PM
To: "Weka machine learning workbench list." <wekalist@list.scms.waikato.ac.nz>
Subject: [Wekalist] exhaustive search: summary results calculation

Dear Weka users,

i'm working with exhaustive search and, looking at the code, i have not clear which is the relationship between the merit figure calculated for the evaluation of each subset within each crossvalidation fold (e.g. the "error rate", i.e. the root mean squared error) and the corresponding value in the results summary. For more clearness, the question comes from the fact that in the classification tasks i'm running (Attribute selected SMO classifier with Exhaustive search and Wrapper evaluator based itself on SMO classifier) the values of RMSE are collected from the folds do not appear to have a direct connection (like averaging or so) with the RMSE value in the results summary table.

Many thanks for your help.

The Wrapper performs an inner 5 fold cross-validation on each training fold (when performing cross-validation evaluation in the Explorer). In fact, it may repeat the cross-validation up to 5 times (and average the results) if the standard deviation is too high.

Cheers,
Mark.