SMHS LinearModeling QC - Revision history

Imoubara at 12:51, 23 May 2016

2016-05-23T12:51:55Z

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-19T13:00:21Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-18T16:30:31Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:37:07Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:23:13Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:19:16Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:17:16Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:15:57Z

‎SMHS Linear Modeling - Quality Control

Pineaumi: /* SMHS Linear Modeling - Quality Control */

2016-05-17T18:15:01Z

‎SMHS Linear Modeling - Quality Control

Imoubara at 14:52, 16 May 2016

2016-05-16T14:52:18Z

@@ Line 4: / Line 4: @@
 '''Questions:
-<li>Is the data what it’s supposed to be (does it represent the study cohort/population)?</li>
+*Is the data what it’s supposed to be (does it represent the study cohort/population)?
-<li>How to inspect the quality of the data?</li>
+*How to inspect the quality of the data?
 Data Quality Control (QC) and Quality Assurance (QA) represent important components of all modeling, analytics, and visualization that precede all subsequent data processing steps. QC and QA may be performed manually or automatically. Statistical quality control involves quantitative methods for monitoring and controlling a process or data derived from observing a natural phenomenon. For example, is there evidence in the plots below of a change in the mean of these processes?

@@ Line 163: / Line 163: @@
   plot.qcc (demo.data.4, 100)
-[[Image:SMHS_LinearModeling_Fig007.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig007.png|500px]]</center>
 Let’s use the "Student's Sleep Data" (sleep) data.

@@ Line 1: / Line 1: @@
 ==[[SMHS_LinearModeling|SMHS Linear Modeling]] - Quality Control ==
-Discussion of data Quality Control (QC) and Quality Assurance (QA) which represent important components of data-driven modeling, analytics, and visualization.
+Discussion of data Quality Control (QC) and Quality Assurance (QA), which represent important components of data-driven modeling, analytics, and visualization.
 '''Questions:
@@ Line 16: / Line 16: @@
      title="Simulation 1", xlab="Index")
-[[Image:SMHS_LinearModeling_Fig003.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig003.png|500px]]</center>
 Now let’s introduce a trend.
@@ Line 29: / Line 29: @@
      title="Simulation 2", xlab="Index")
-[[Image:SMHS_LinearModeling_Fig004.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig004.png|500px]]</center>
 Our goal is to use statistical quality control to automatically identify issues with the data.  The qcc package in R provides methods for statistical quality control – given the data, it identifies candidate points as outliers based on the Shewhart Rules. Color-coding the data also helps point out irregular points.
@@ Line 61: / Line 61: @@
   plot(MyQC) # , chart.all=FALSE)
-[[Image:SMHS_LinearModeling_Fig005.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig005.png|500px]]</center>
   # add warning limits at 2 std. deviations
@@ Line 75: / Line 75: @@
   Conf= a numeric value used to compute control limits, specifying the number of standard deviations (if conf > 1) or the confidence level (if 0 < conf < 1)
-[[Image:SMHS_LinearModeling_Fig006.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig006.png|500px]]</center>
 Natural processes may have errors that are non-normally distributed. However, using (appropriate) transformations we can often normalize the errors.
@@ Line 176: / Line 176: @@
   obj_avg_test_train <-  qcc(q[,1:5],  type="xbar", newdata=q[,6:10])
-[[Image:SMHS_LinearModeling_Fig008.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig008.png|500px]]</center>
   # How is this different from this?
@@ Line 186: / Line 186: @@
 plots the X chart for training and testing (11-20) sleep data where the statistics and the control limits are based on the first 10 (training) samples.
-[[Image:SMHS_LinearModeling_Fig009.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig009.png|500px]]</center>
   Now try (range) QCC plot
@@ Line 226: / Line 226: @@
   # oc.curves(obj_test_1_10_train_11_20, identify=TRUE) # to manually identify specific OC points
-[[Image:SMHS_LinearModeling_Fig11.png|500px]]
+<center>[[Image:SMHS_LinearModeling_Fig11.png|500px]]</center>
 The function oc.curves returns a matrix or a vector of probabilities values representing the type II errors for different sample-sizes. See help (oc.curves), e.g., identify=TRUE, which allows to interactively identify values on the plot, for all options.

@@ Line 181: / Line 181: @@
   obj_avg_new <-  qcc(q[,1:10],  type="xbar")
-This control chart has the solid horizontal line (center), the upper and lower control limits (dashed lines), and the sample group statistics (e.g., mean) are drawn as a piece-wise line connecting the points. The bottom of the plot includes summary statistics and the number of points beyond control limits and the number of violating runs. If the process is "in-control", we can use estimated limits for monitoring prospective (new) data sampled from the same process/protocol. For instance,
+This control chart has the solid horizontal line (center), the upper and lower control limits (dashed lines), and the sample group statistics (e.g., mean; drawn as a piece-wise line connecting the points). The bottom of the plot includes summary statistics and the number of points beyond control limits and the number of violating runs. If the process is "in-control", we can use estimated limits for monitoring prospective (new) data sampled from the same process/protocol. For instance,
   obj_test_1_10_train_11_20  <-  qcc(sleep[1:10,],  type="xbar", newdata=sleep[11:20,])
@@ Line 191: / Line 191: @@
   obj_R  <-  qcc(q[,1:5],  type="R", newdata=q[,6:10])
-A control chart aims to enhance our ability to monitor and track a process proxied by the data. When special causes of variation (random or not) are present the data may be considered "out of control". Corresponding action may need to be taken to identify, control for, or eliminate such causes. A process is declared to be "controlled" if the plot of all data points are randomly spread out within the control limits. These Lower and Upper Control Limits (LCL, UCL) are usually computed as ±3s from the center (e.g., mean). This QCC default limits can be changed using the argument nsigmas or by specifying the confidence level via the confidence level  argument.
+A control chart aims to enhance our ability to monitor and track a process proxied by the data. When special causes of variation (random or not) are present, the data may be considered "out of control". Corresponding action may need to be taken to identify, control for, or eliminate such causes. A process is declared to be "controlled" if the plot of all data points are randomly spread out within the control limits. These Lower and Upper Control Limits (LCL, UCL) are usually computed as ±3s from the center (e.g., mean). The QCC default limits can be changed using the argument nsigmas or by specifying the confidence level via the confidence level  argument.
 <center>
@@ Line 211: / Line 211: @@
 |"np"||The number of nonconforming units is plotted.  Control limits are based on the binomial distribution.
 |-
-|"c"||The number of defectives per unit are plotted. This chart assumes that defects of t he quality attribute are rare, and the control limits are computed based on the Poisson distribution.
+|"c"||The number of defectives per unit are plotted. This chart assumes that defects of the quality attribute are rare, and the control limits are computed based on the Poisson distribution.
 |-
-|"u"||The average number of defectives per unit is plotted. The Poisson distribution is used to compute control limits, but,unlike the c chart, this chart does not require a constant number of units.
+|"u"||The average number of defectives per unit is plotted. The Poisson distribution is used to compute control limits; but, unlike the c chart, this chart does not require a constant number of units.
 |}
 </center>
-When the process is governed by a Gaussian distribution the ±3s limits correspond to a two-tails probability of p=0.0027.
+When the process is governed by a Gaussian distribution, the ±3s limits correspond to a two-tails probability of p=0.0027.
-Finally, an operating characteristic (OC) curve shows the probability of not detecting a shift in the process (false-negative, type II error), i.e., the probability of erroneously accepting a process as being "in control", when in fact, it’s out of control.
+Finally, an operating characteristic (OC) curve shows the probability of not detecting a shift in the process (false-negative, type II error); i.e., the probability of erroneously accepting a process as being "in control", when in fact, it’s out of control.
   # par(mfrow=c(1,1))
@@ Line 228: / Line 228: @@
 [[Image:SMHS_LinearModeling_Fig11.png|500px]]
-The function oc.curves returns a matrix or a vector of probabilities values representing the type II errors for different sample-sizes. See help(oc.curves), e.g., identify=TRUE, which allows to interactively identify values on the plot, for all options.
+The function oc.curves returns a matrix or a vector of probabilities values representing the type II errors for different sample-sizes. See help (oc.curves), e.g., identify=TRUE, which allows to interactively identify values on the plot, for all options.
 Notice that the OC curve is "S"-shaped. As expected, this is because as the percent of non-conforming values increases, the probability of acceptance decreases. A small sub-sample, instead of inspecting the entire data, may be used to determine the quality of a process. We can accept the data as in-control as long as the process percent nonconforming is below a predefined level.

@@ Line 35: / Line 35: @@
 The Shewhart control charts rules (cf. 1930’s) are based on monitoring events that are unlikely when the controlled process is stable. Incidences of such atypical events are alarm signals, suggesting that stability of the process may be compromised and the process is changed.
-An instance of such an unlikely event is the situation when the upper/lower control limits (UCL or LCL) are exceeded. UCL and LCL are constructed as ±3? limits, indicating that the process is under control within them. Additional warning limits (LWL and UWL) are constructed at ±2? or ±?. Other rules specifying events having low probability when the process is under control can be constructed:
+An instance of such an unlikely event would be the situation in which the upper/lower control limits (UCL or LCL) are exceeded. UCL and LCL are constructed as ±3? limits, indicating that the process is under control within them. Additional warning limits (LWL and UWL) are constructed at ±2? or ±?. Other rules specifying events having low probability when the process is under control can be constructed:
 .  One point exceeds LCL/UCL.

@@ Line 7: / Line 7: @@
 <li>How to inspect the quality of the data?</li>
-Data Quality Control (QC) and Quality Assurance (QA) represent important components of all modeling, analytics and visualization that precede all subsequent data processing steps. QC and QA may be performed manually or automatically. Statistical quality control involves quantitative methods for monitoring and controlling a process or data derived from observing a natural phenomenon. For example, is there evidence in the plots below of a change in the mean of these processes?
+Data Quality Control (QC) and Quality Assurance (QA) represent important components of all modeling, analytics, and visualization that precede all subsequent data processing steps. QC and QA may be performed manually or automatically. Statistical quality control involves quantitative methods for monitoring and controlling a process or data derived from observing a natural phenomenon. For example, is there evidence in the plots below of a change in the mean of these processes?
   # simulate data with base value of 100 w/ normally distributed error