Re: [Wekalist] ROC curve detailed help
by Hans van Rijnberk , Assort Vision, Utrecht
In the Gui version e.g. chooce explorer, load your data and perform e
classification via classify. Right click on the result in the results list.
A list with performance stats will occur. Chooce Thresholdcurve and select
true or false which ever class is youre "golden standard".
You get a panel chooce the x and y axis selections TP rate and FP rate. This
shows the ROC. To safe all threshold data in orde to export them click the
save button and provide a name. You will get a arff file with a simple header.
There is one problem however weka throws away one thershold point assuming
that with N instances N-1 threshold points can be distinguished. This is not
true because accepting all and rejecting all are true alternatieves in this
analysis. So with N instances N threshold can be distinguished. It is a
pitty that these threshold curve data don't provide the actual class because
with that it is very easy to calculate the area under the ROC curve with
Mann-Whitney U statistic, which is the exact way SPSS calculates it though
ther is an exact parametric alternative (from Metz).
Alternatively you can run the callClassifier program from
http://www.oefai.at/~alexsee/WEKA/ (see Appendix A) which will for each
instance produce a probability that it belongs to each of the alternative
classes together with the actual class. It is easy to produce ROC)'s) from
these data e.g. in a spreadsheet.
See earlier mails in the list among whixh one in which I explained the way
how to calculate the area under the ROC.
At 07:40 24-2-2004 -0800, Uma Chen wrote:
>I have created the classification model in WEKA using
>SMO with F-value ~78. Now I want to draw the ROC
>curve. I need the detailed help whether It is possible
>in WEKA or how I will do the same with other
>Do you Yahoo!?
>Yahoo! Mail SpamGuard - Read only the mail you want.
>Wekalist mailing list
Hans van Rijnberk
Assort Vision (machine vision software & information services)
3524 KM Utrecht,
031 (0)30 2148681 / 2889531