Re: [Wekalist] ROC Analysis in WEKA
by Hans van Rijnberk , Assort Vision, Utrecht
A short reply:
TP rate and FP rate in ThresholdCurve constitute the ROC. An N-instances
testset results in a ROC with a maximum of N-2 points because weka's ROC
the points (FP%,TP%) = (0,0) and (FP%,TP%) = (1,1). However they should, so
include them. These are crucial points!
The N-2 points are decreased when particular threshold values in the set
occur more then once. AUC is calculated by use of the trapezoid rule which
gives the same results as by the Mann Whitney statistic.
What lacks is a bootstrap and permutation facility for testing and setting
confidence intervals of Threshold statistics.
Since this is not so difficult. We might implement it and post it.
At 14:58 3/2/05 -0700, lei tang wrote:
>You can check weka.classifiers.evaluation.ThresholdCurve.
> In this class, you can get the AUC. But I am not sure about how to
>generate a good ROC curve by weka. In explorer version, you can
>generate a ROC for a classifier. But that graph is more like several
>points rather than "a curve".
>On Tue, 01 Feb 2005 12:51:22 +1100, Tarkan Kurt <tarkan38(a)hotmail.com> wrote:
>> Hi everyone,
>> Sorry if this question has been covered before, but could anyone help in how
>> to perform roc analysis in weka. Simple or Advanced, all help is welcome.
>> Thank you
>> Wekalist mailing list
>Wekalist mailing list
Hans van Rijnberk