SMHS PCA ICA FA - Revision history

Dinov: /* Applications */

2014-11-24T16:39:04Z

‎Applications

Dinov: /* Fundamentals */

2014-11-24T14:23:47Z

‎Fundamentals

Dinov: /* Fundamentals */

2014-11-24T13:01:17Z

‎Fundamentals

Dinov: /* Fundamentals */

2014-11-24T12:53:01Z

‎Fundamentals

Dinov: /* Fundamentals */

2014-11-24T03:38:19Z

‎Fundamentals

Dinov: /* Fundamentals */

2014-11-24T03:17:20Z

‎Fundamentals

Dinov: /* Theory */

2014-11-24T03:12:13Z

‎Theory

Dinov: /* Parkinson's disease case study */

2014-11-23T21:21:09Z

‎Parkinson's disease case study

Dinov: /* Parkinson's disease case study */

2014-11-23T20:07:44Z

‎Parkinson's disease case study

Dinov: /* Parkinson's disease case study */

2014-11-23T19:52:11Z

‎Parkinson's disease case study

@@ Line 264: / Line 264: @@
 The table below includes data for N=33 Parkinson's disease (PD) cases (3001, …, 3808) from the [http://www.ppmi-info.org/ Parkinson’s Progression Markers Initiative (PPMI) database]. We are interested in examining the relationships between imaging biomarkers, derived from the red nucleus in the Substantia Nigra (SN), and [http://www.movementdisorders.org/MDS-Files1/PDFs/MDS-UPDRS-Rating-Scales/NewUPDRS7308final.pdf MDS-UPDRS assessment measures] for Parkinson's Diseases patients, Part I.A, Part I.B, Part II and Part III scores.
-The derived imaging biomarkers include:
+* '''Hypothesis''': The a priori hypothesis is that there is a strong inverse correlation between the imaging biomarkers (voxel intensity ratios) and the MDS-UPDRS scores. In other words, an increase of the (average) voxel intensities (i.e., more intact nigrosome-1) would correspond with a decrease in UPDRS scores (i.e., less PD symptoms), and vice-versa, lower voxel intensity ratios would correspond with higher UPDRS scores.
-The Movement Disorder Society (MDS)-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): includes Part I (Non-motor Experiences of Daily Living); Part II (Motor Experiences of Daily Living); Part III (Motor Examination); and Part IV (Motor Complications) which was excluded as it was only conducted for PPMI subjects who had started PD medication.
+* '''Data''':
 <center>
@@ Line 343: / Line 346: @@
 </center>
-R-script analyses of the PPMI data:
+* R-script analyses of the PPMI data:
   dataset <- read.csv('C:\\Users\\Desktop\\PPMI_data.csv', header = TRUE)
@@ Line 455: / Line 458: @@
   Proportion Var    0.40    0.24
   Cumulative Var    0.40    0.64
 * [[SOCR_EduMaterials_AnalysisActivities_PCA| This SOCR Activity]] demonstrates the utilization of a SOCR analysis package for statistical computing in the SOCR environment. It presents a general introduction to PCA and the theoretical background of this statistical tool and demonstrates how to use PCA and read and interpret the outcome. It introduces students to inputting data in the correct format, reading the results of PCA and interpreting the resulting transformed data.

@@ Line 11: / Line 11: @@
 ===Theory===
 ====Fundamentals====
-First review the [https://en.wikipedia.org/wiki/Matrix_multiplication rules for matrix and vector multiplication]. In an essence, the arithmetic process of multiplying numbers in row $i$ in matrix, or vector, $X^T$ and column $j$ in matrix, or vector, $A$ corresponds to component-wise multiplication followed by adding the terms, summation, to obtain entry $X\times A_{i,j}$ in the product matrix. As an example, $X_{1\times n}^T A_{n\times n}X_{n\times 1} = B_{1\times 1}$ is a scalar value. The transpose notation, $X^T$, indicates that the rows and columns are swapped, e.g., for a standard vector represented as a column, $X=\begin{pmatrix}X_1\\X_2\\...\\X_N \end{pmatrix}$, the transpose is the same vector as a row, $X^T=(X_1, X_2, ..., X_N)$.
+First review the [http://en.wikipedia.org/wiki/Matrix_multiplication rules for matrix and vector multiplication]. In an essence, the arithmetic process of multiplying numbers in row $i$ in matrix, or vector, $X^T$ and column $j$ in matrix, or vector, $A$ corresponds to component-wise multiplication followed by adding the terms, summation, to obtain entry $X\times A_{i,j}$ in the product matrix. As an example, $X_{1\times n}^T A_{n\times n}X_{n\times 1} = B_{1\times 1}$ is a scalar value. The transpose notation, $X^T$, indicates that the rows and columns are swapped, e.g., for a standard vector represented as a column, $X=\begin{pmatrix}X_1\\X_2\\...\\X_N \end{pmatrix}$, the transpose is the same vector as a row, $X^T=(X_1, X_2, ..., X_N)$.
 Now, suppose we have a sample of $n$ observations as points in a plane: $\{p_1, p_2, …, p_n\}$ in the 2D space, $p=(x_1,x_2)$. How can we account (approximately) for the variation in the sample in as few variables as possible. The figure below illustrates the geometric nature of '''principal components (PCs)'''.
@@ Line 47: / Line 47: @@
 :: $a_2^Ta_2 = 1.$
-Thus, for a pair of (Lagrange multipliers) $\lambda_1$ and $\lambda_2$, we need to maximize:
+Thus, for a pair of ([http://en.wikipedia.org/wiki/Lagrange_multiplier Lagrange multipliers]) $\lambda_1$ and $\lambda_2$, we need to maximize:
 $$ a_1^TSa_2 -\lambda_1(a_2^Ta_2-1) - \lambda_2(a_1^Ta_2).$$
 Differentiating w.r.t. $a_1$ and setting the derivative equal to zero yields that:

@@ Line 11: / Line 11: @@
 ===Theory===
 ====Fundamentals====
-First review the [https://en.wikipedia.org/wiki/Matrix_multiplication rules for matrix and vector multiplication]. In an essence, the arithmetic process of multiplying numbers in row $i$ in matrix, or vector, $X$ and column $j$ in matrix, or vector, $A$ corresponds to component-wise multiplicaition followed by adding the terms, summation, to obtain entry $X\times A_{i,j}$ in the product matrix. As an example, $X_{1\times n} A_{n\times n}X_{n\times 1} = B_{1\times 1}$ is a scalar value.
+First review the [https://en.wikipedia.org/wiki/Matrix_multiplication rules for matrix and vector multiplication]. In an essence, the arithmetic process of multiplying numbers in row $i$ in matrix, or vector, $X^T$ and column $j$ in matrix, or vector, $A$ corresponds to component-wise multiplication followed by adding the terms, summation, to obtain entry $X\times A_{i,j}$ in the product matrix. As an example, $X_{1\times n}^T A_{n\times n}X_{n\times 1} = B_{1\times 1}$ is a scalar value. The transpose notation, $X^T$, indicates that the rows and columns are swapped, e.g., for a standard vector represented as a column, $X=\begin{pmatrix}X_1\\X_2\\...\\X_N \end{pmatrix}$, the transpose is the same vector as a row, $X^T=(X_1, X_2, ..., X_N)$.
 Now, suppose we have a sample of $n$ observations as points in a plane: $\{p_1, p_2, …, p_n\}$ in the 2D space, $p=(x_1,x_2)$. How can we account (approximately) for the variation in the sample in as few variables as possible. The figure below illustrates the geometric nature of '''principal components (PCs)'''.

@@ Line 11: / Line 11: @@
 ===Theory===
 ====Fundamentals====
-Suppose we have a sample of $n$ observations as points in a plane: $\{p_1, p_2, …, p_n\}$ in the 2D space, $p=(x_1,x_2)$. How can we account (approximately) for the variation in the sample in as few variables as possible. The figure below illustrates the geometric nature of '''principal components (PCs)'''.
+First review the [https://en.wikipedia.org/wiki/Matrix_multiplication rules for matrix and vector multiplication]. In an essence, the arithmetic process of multiplying numbers in row $i$ in matrix, or vector, $X$ and column $j$ in matrix, or vector, $A$ corresponds to component-wise multiplicaition followed by adding the terms, summation, to obtain entry $X\times A_{i,j}$ in the product matrix. As an example, $X_{1\times n} A_{n\times n}X_{n\times 1} = B_{1\times 1}$ is a scalar value.
 <center>

@@ Line 28: / Line 28: @@
 To find the transformation ($a$) we express the $var(Z_1)$ in terms of the variance-covariance matrix of $X$:
-:: $var(Z_1) = E(Z_1^2) – (E (Z_1))^2 = \sum _{i,j,=1}^N {a_{I,1} a_{j,1} E(x_i x_j)} –
+:: $var(Z_1) = E(Z_1^2) – (E (Z_1))^2 = \sum _{i,j,=1}^N {a_{i,1} a_{j,1} E(x_i x_j)} –
 \sum _{i,j,=1}^N { a_{i,1} a_{j,1} E(x_i)E( x_j)} $ $= \sum _{i,j,=1}^N { a_{i,1} a_{j,1} S_{i,j}}$,
 where $ S_{i,j} = \sigma_{x_i x_j} = E(X_i X_j) – E(X_i)E(X_j)$.
-Thus, :$var(Z_1) =a_1^TSa_1$, where $S=(S_{I,j})$ is the variance covariance matrix of $X=\{X_1, …, X_N\}$.
+Thus,
 As $a_1$ maximizes $Var(Z_1)$ subject to $||a_1||=a_i^Ta_1=1$, we set up this optimization problem as:
-$\max_{a_1} \{a_1^T Sa_1 - \lambda(a_1^Ta_1 -1) \}$.
+$\max_{a_1} \{a_1^T Sa_1 - \lambda(a_1^Ta_1 -1) \}.$
 Differentiating w.r.t. $a_1$ and setting the derivative equal to zero, yields:
 $Sa_1 - \lambda a_1 = 0$, and thus, $(S-\lambda I_N)a_1=0$.
@@ Line 42: / Line 43: @@
 Similarly, the second eigenvector $a_2$ maximizes $Var(Z_2)$, subject to:
 :: $0=Cov(Z_2,Z_1)=a_1^TSa_2=\lambda_1a_1^Ta_2$, and
-:: $a_2^Ta_1 = 1.$
+:: $a_2^Ta_2 = 1.$
 Thus, for a pair of (Lagrange multipliers) $\lambda_1$ and $\lambda_2$, we need to maximize:
-$$ a_1^TSa_2 -\lambda_1(a_2^Ta_2-1) - \lambda_2(a_2^Ta_1)$$.
+$$ a_1^TSa_2 -\lambda_1(a_2^Ta_2-1) - \lambda_2(a_1^Ta_2).$$
 Differentiating w.r.t. $a_1$ and setting the derivative equal to zero yields that:
-$(S-\lambda_2 I_N)a_2^T=0$, which means that $a_2$ is another eigenvector of $S$ corresponding to the eigenvalue $\lambda_2$. And so on.
+$(S-\lambda_2 I_N)a_2=0$, which means that $a_2$ is another eigenvector of $S$ corresponding to the eigenvalue $\lambda_2$. And so on.
-In genetal, $Var(Z_k)=a_k^TSa_k=\lambda_k$, where $a_k$ is the $k^{th}$ largest eigenvalue of $S$ and $Z_k$ is the $k^{th}$ PC retaining the $k^{th}$ largest variation in the sample $X$.
+In general, $Var(Z_k)=a_k^TSa_k=\lambda_k$, where $a_k$ is the $k^{th}$ largest eigenvalue of $S$ and $Z_k$ is the $k^{th}$ PC retaining the $k^{th}$ largest variation in the sample $X$.
 In Matrix notation:
 :: Observations $X=\{X_1, X_2, …, X_N \}$
-:: PCs: $Z=\{Z_1, Z_2, …, Z_N \}$, with $Z=A^TZ$ and $A=\{a_1, a_2, …, a_N\}$ is an orthogonal $N\times N$ matrix with columns representing the eigenvectors of S. That is, $\Lambda = A^TSA$, where $\Lambda$ is a diagonal matrix $\Lambda=diag(\lambda_1, \lambda_2, …, \lambda_n)$.
+:: PCs: $Z=\{Z_1, Z_2, …, Z_N \}$, with $Z=A^TZ$ and $A=\{a_1, a_2, …, a_N\}$ is an orthogonal $N\times N$ matrix with columns representing the eigenvectors of S. That is, $\Lambda = A^T S A$, where $\Lambda$ is a diagonal matrix $\Lambda=diag(\lambda_1, \lambda_2, …, \lambda_n)$ of eigenvalues.
 ====PCA (principal component analysis)====

@@ Line 352: / Line 352: @@
   # PCA/FA Analyses
-  # pca.model <- princomp(dataset, cor=TRUE)
+  par(mfrow=c(1,1))
   summary(pca.model) # print variance accounted for
-  loadings(pca.model) # pc loadings
+  loadings(pca.model) # pc loadings (i.e., igenvector columns)
   Loadings:
@@ Line 362: / Line 363: @@
   Part_IA                           0.383 -0.373  0.710 -0.320	  0.238  	0.227
   Part_IB                           0.460        -0.112  0.794	  0.292  	0.226
-  Part_II                           0.425 -0.342 -0.464 -0.262   -0.534  	0.365
+  Part_II                           0.425 -0.342 -0.464 -0.262     -0.534  	0.365
-  Part_III                          0.498                        -0.183 	-0.844
+  Part_III                          0.498                          -0.183 	-0.844
   plot(pca.model, type="lines") # scree plot

@@ Line 346: / Line 346: @@
   summary(Fit_III)
-  par(mfrow=c(2,2))
+  par(mar = rep(3, 4))
-  plot(Fit_III)
+ plot(Fit_IA); # text(0.5,0.5,"First title",cex=2,font=2)
   # PCA/FA Analyses

@@ Line 317: / Line 317: @@
   # some diagnostic plots
   par(mfrow=c(2,3))
-  plot(sort(dataset$Top_of_SN_Voxel_Intensity_Ratio))
+  plot(sort(dataset$\$ $Top_of_SN_Voxel_Intensity_Ratio))
-  hist(dataset$Top_of_SN_Voxel_Intensity_Ratio)
+  hist(dataset$\$ $Top_of_SN_Voxel_Intensity_Ratio)
-  plot(density(dataset$Top_of_SN_Voxel_Intensity_Ratio,na.rm=FALSE))
+  plot(density(dataset$\$ $Top_of_SN_Voxel_Intensity_Ratio,na.rm=FALSE))
-  plot(sort(dataset$Side_of_SN_Voxel_Intensity_Ratio))
+  plot(sort(dataset$\$ $Side_of_SN_Voxel_Intensity_Ratio))
-  hist(dataset$Side_of_SN_Voxel_Intensity_Ratio)
+  hist(dataset$\$ $Side_of_SN_Voxel_Intensity_Ratio)
-  plot(density(dataset$ Side_of_SN_Voxel_Intensity_Ratio,na.rm=FALSE))
+  plot(density(dataset$\$ $Side_of_SN_Voxel_Intensity_Ratio,na.rm=FALSE))
   # Try MANOVA first to explore predictive value of imaging biomarkers on clinical PD scores
@@ Line 365: / Line 365: @@
   plot(pca.model, type="lines") # scree plot
-  pca.model$scores # the principal components
+  pca.model$\$ $scores # the principal components
   par(mfrow=c(1,1))
   biplot(pca.model)
@@ Line 378: / Line 378: @@
   print(fa.model, digits=2, cutoff=.3, sort=TRUE)
   # plot factor 1 by factor 2
-  load <- fa.model$loadings[,1:2]
+  load <- fa.model$\$ $loadings[,1:2]
   plot(load,type="n") # set up plot
   text(load, labels=names(fa.model),cex=.7) # add variable names
-# factanal(x = dataset, factors = 2, rotation = "varimax")
+ # factanal(x = dataset, factors = 2, rotation = "varimax")
   Uniquenesses: