SMHS ROC - Revision history

Dinov: /* Example 2: Comparing Classifiers for Alzheimer’s Disease */

2026-02-23T13:42:32Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2026-02-23T13:39:33Z

‎Constructing the ROC Curve

2026-02-23T13:34:48Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2026-02-19T18:44:33Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2026-02-10T21:16:18Z

‎References

2025-12-05T22:02:18Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2025-12-05T22:01:30Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2025-12-05T21:42:32Z

‎Example 2: Comparing Classifiers for Alzheimer’s Disease

2025-12-05T21:36:04Z

2014-11-23T18:21:24Z

‎R AUC/ROC Example

@@ Line 133: / Line 133: @@
 ==== Example 2: Comparing Classifiers for Alzheimer’s Disease ====
-This example compares Random Forest and Logistic Regression models using [https://wiki.socr.umich.edu/index.php/SOCR_Data_July2009_ID_NI#Curvedness_Data Global Gray Matter Volume (GMV) and demographic features].
+This example compares [https://socr.umich.edu/DSPA2/DSPA2_notes/06_ML_NN_SVM_RF_Class.html Random Forest (machine learning decision/prediction)] and [https://socr.umich.edu/DSPA2/DSPA2_notes/11_FeatureSelection.html#216_Logistic_Transformation Logistic Regression models] using [https://wiki.socr.umich.edu/index.php/SOCR_Data_July2009_ID_NI#Curvedness_Data Global Gray Matter Volume (GMV) and demographic features].
 <pre>

@@ Line 61: / Line 61: @@
 * Cut-point = 5 (strict):
-  T4 ≤ 5 → test positive
-   <math>\text{Sensitivity} = 18/32 = 0.56</math>,
+<math>T4 \leq 5 \to \text{ test positive }, </math>
-  <math>\text{Specificity} = 92/93 = 0.99</math>
 * Cut-point = 7 (moderate):
-  <math>\text{Sensitivity} = 25/32 = 0.78</math>,
-   <math>\text{Specificity} = 75/93 = 0.81</math>
+<math>\text{Sensitivity} = 25/32 = 0.78,</math>
 * Cut-point = 9 (lenient):
-  <math>\text{Sensitivity} = 29/32 = 0.91</math>,
-  <math>\text{Specificity} = 39/93 = 0.42</math>
+<math>\text{Sensitivity} = 29/32 = 0.91,</math>
 Plotting Sensitivity (y-axis) versus FPR = <math>1 - \text{Specificity}</math> (x-axis) for all thresholds yields the ROC curve.

@@ Line 126: / Line 126: @@
 ==== Example 2: Comparing Classifiers for Alzheimer’s Disease ====
-This example compares Random Forest and Logistic Regression models using [[https://wiki.socr.umich.edu/index.php/SOCR_Data_July2009_ID_NI#Curvedness_Data | Global Gray Matter Volume (GMV) and demographic features]].
+This example compares Random Forest and Logistic Regression models using [https://wiki.socr.umich.edu/index.php/SOCR_Data_July2009_ID_NI#Curvedness_Data Global Gray Matter Volume (GMV) and demographic features].
 <pre>

@@ Line 258: / Line 258: @@
 plot(perf_glm, col = "red", lwd = 2, add = TRUE)
 abline(a = 0, b = 1, lty = 2, col = "gray")
 legend("bottomright",
-        legend = c(paste("Random Forest (AUC =", round(auc_rf, 3), ")"),
+        legend = c(paste("RF (AUC =", round(auc_rf, 3), ")"),
-                   paste("Logistic Regression (AUC =", round(auc_glm, 3), ")")),
+                   paste("GLM (AUC =", round(auc_glm, 3), ")")),
-        col = c("blue", "red"), lwd = 2, cex = 0.9, box.lty = 0)
+        col = c("blue", "red"),
+       lwd = 2,
-# Add confidence intervals for AUC
+       cex = 0.75,          # Reduced size (0.9 was likely too big)
-cat("\n=== AUC with 95% Confidence Intervals ===\n")
+       bty = "n",           # "n" removes the box entirely for a cleaner look
-cat("Random Forest: ", round(ci.auc(roc_rf)[1], 3), "-",
+       inset = c(0.02, 0.02), # Nudges the legend slightly away from the axes
-    round(ci.auc(roc_rf)[3], 3), "\n")
+       y.intersp = 0.8)     # Tightens the vertical spacing between lines
 </pre>

@@ Line 330: / Line 330: @@
 === References ===
 * Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. *Radiology*. 1982;143(1):29–36.
 * Metz CE. Basic principles of ROC analysis. *Seminars in Nuclear Medicine*. 1978;8(4):283–298.
@@ Line 336: / Line 338: @@
 <hr>
-* SOCR Home page: http://www.socr.umich.edu
+* SOCR Home page: https://socr.umich.edu
-{{translate|pageName=http://wiki.socr.umich.edu/index.php?title=SMHS_ROC}}
+{{translate|pageName=https://wiki.socr.umich.edu/index.php?title=SMHS_ROC}}

@@ Line 129: / Line 129: @@
 <pre>
 library(randomForest)
 library(ROCR)
-# Load and preprocess data (binary: AD vs. Non-AD)
+dataset <- html_table(html_nodes(wiki_url, "table")[[2]])
-dataset <- read.csv('GMV.csv')
-dataset_clean <- subset(dataset, Group != "MCI")  # Exclude MCI for clarity
+# Dynamically load GMV dataset from SOCR GitHub
-dataset_clean$Group <- factor(dataset_clean$Group)
+# url <- "https://raw.githubusercontent.com/SOCR/SOCR_Data/master/CSV_SOCR_Data/CSV_July2009_ID_NI_Curvedness_Data.csv"
 # Random Forest model
 set.seed(123)
 rf_model <- randomForest(Group ~ GMV + Age + Sex, data = dataset_clean, ntree = 100)
-rf_probs <- predict(rf_model, dataset_clean, type = "prob")[,2]
+rf_probs <- predict(rf_model, dataset_clean, type = "prob")[, "AD"]  # Explicitly reference "AD" column
-# Logistic regression model
+# Logistic regression model (note: Sex may be excluded due to collinearity or design)
 glm_model <- glm(Group ~ GMV + Age, data = dataset_clean, family = binomial)
 glm_probs <- predict(glm_model, dataset_clean, type = "response")

@@ Line 307: / Line 307: @@
 Below is an example using the ROCR R-package to generate the ROC curve and compute the AUC. First download the data table: [[SOCR_Data_July2009_ID_NI#Curvedness_Data|Alzheimer's disease (AD) dataset of Global Gray Matter Volume (GMV) and Cortical Surface Curvedness morphometry measures]], as CSV file (C:\User\GMV.csv) on your hard-drive.
   #install.packages("randomForest")
-  #library("randomForest")
+  library("randomForest")
   dataset <- read.csv('C:\\Users\\GMV.csv')
@@ Line 322: / Line 325: @@
   AD.rf <-randomForest(Group ~ MMSE+CDR+Sex+Age+TBV+GMV+WMV+CSFV+Curvedness, ntree=100, keep.forest=TRUE, importance=TRUE)
   AD.rf.predict <- predict(AD.rf, dataset, type="prob")[,2]
   plot(performance(prediction(AD.rf.predict, dataset$\$ $Group),"tpr","fpr"),col = "blue")