On 22/06/2009, at 2:10 PM, Eduardo Girardi wrote:
I need urgent help on a question about ROC Curves.
I've been reading in various places about how to compose a ROC
Curve, and one description that I found was: "Weka's ROC curves plot
correspond to the number of TPs and FPs that result from setting
various thresholds on the probability of the positive class." By
But how in fact WEKA does this variation on thresholds in, both,
MultiLayer Perceptron and J48 algorithms? Which part of the
calculation WEKA changes to compose varios cut-points (threshoulds)?
The process works in the same way for every classifier. Predictions
are collected from the classifier for each test instance (in the case
of 10-fold cross validation, the predictions are collected from each
test fold). This set of predictions are then sorted in descending
order of probability assigned to the positive class. The curve is
generated by stepping down this list (starting from the top) and
noting how many true positives and false positives are contained in
the list up to that point. The threshold at each point is just the
probability of the true class at that point in the sorted list.
Senior Developer/Consultant, Pentaho Open Source Business Intelligence
Citadel International, Suite 340, 5950 Hazeltine National Dr.,
Orlando, FL 32822, USA
+64 7 847-3537 office, +64 21 399-132 mobile, +1 815 550-8637 fax,
Skype: mark.andrew.hall, Yahoo: mark_andrew_hall
Download the latest release today <http://www.sourceforge.net/projects/pentaho