SMHS DataManagement - Revision history

Zhenxunw: /* References */

2014-09-02T18:26:31Z

‎References

Zhenxunw: /* Theory */

2014-09-02T18:25:52Z

‎Theory

Liyufang: /* References */

2014-08-29T16:57:17Z

‎References

Liyufang: /* References */

2014-08-29T16:56:59Z

‎References

Dinov at 19:44, 5 August 2014

2014-08-05T19:44:55Z

Clgalla: /* == */

2014-08-04T15:48:06Z

‎==

Clgalla: /* == */

2014-08-04T15:43:24Z

‎==

Clgalla at 15:42, 4 August 2014

2014-08-04T15:42:53Z

Clgalla: /* Theory */

2014-08-04T15:36:06Z

‎Theory

Clgalla: /* Theory */

2014-08-01T21:23:25Z

‎Theory

@@ Line 268: / Line 268: @@
 *[http://en.wikipedia.org/wiki/Data_warehouse  Data Warehouse Wikipedia]
 *[http://en.wikipedia.org/wiki/Database  Database Wikipedia]
 *[http://en.wikipedia.org/wiki/Array_data_structure  Array Data Structure Wikipedia]
 *[http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html  Data Handling, Responsible Conduct in Data Management]

@@ Line 122: / Line 122: @@
    3.4     2 10
-'''Binary ASCII:'''  ASCII, which is short for American Standard Code for Information Interchange is a character-encoding scheme originally based on English alphabet that encodes 128 specified characters – the numbers 0 – 9, the letters a – z and A – Z, some basic punctuation symbols, some control codes that originated with teletype machines and a bland space – into the 7-bit binary integers. It represents text in computers, communications equipment, and other devices that use text. To convert the ASCII code to binary character (part):
+'''Binary ASCII:'''  [http://en.wikipedia.org/wiki/Array_data_structure ASCII], which is short for American Standard Code for Information Interchange is a character-encoding scheme originally based on English alphabet that encodes 128 specified characters – the numbers 0 – 9, the letters a – z and A – Z, some basic punctuation symbols, some control codes that originated with teletype machines and a bland space – into the 7-bit binary integers. It represents text in computers, communications equipment, and other devices that use text. To convert the ASCII code to binary character (part):
 <center>
 {|class="wikitable" style="text-align:center; width:75%" border="1"
@@ Line 181: / Line 181: @@
 </center>
-'''Handling:''' the process of ensuring that the research data is stored, archived or disposed off in a safe and secure manner during and after the conclusion of a research project. This includes the development of policies and procedures to manage data handled electronically as well as through non-electronic means.
+'''[http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html Handling:]''' the process of ensuring that the research data is stored, archived or disposed off in a safe and secure manner during and after the conclusion of a research project. This includes the development of policies and procedures to manage data handled electronically as well as through non-electronic means.
 *Data handling is important in ensuring the integrity of research data since it addresses concerns related to confidentially, security, and preservation/retention of research data. Proper planning for data handling can also result in efficient and economical storage, retrieval, and disposal of data. In the case of data handled electronically, data integrity is a primary concern to ensure that recorded data is not altered, erased, lost or accessed by unauthorized users.

@@ Line 269: / Line 269: @@
 *[http://en.wikipedia.org/wiki/Database  Database Wikipedia]
 *[http://en.wikipedia.org/wiki/Array_data_structure  Array Data Structure Wikipedia]
-*[http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html  Data Handling, Responsible Conduct in Data Management, online resource]
+*[http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html  Data Handling, Responsible Conduct in Data Management]

@@ Line 263: / Line 263: @@
 ===References===
-*[http://mirlyn.lib.umich.edu/Record/004199238 Statistical inference / George Casella, Roger L. Berger]
+*[http://en.wikipedia.org/wiki/Table_(information)  Table (Information) Wikipedia]
-*[http://mirlyn.lib.umich.edu/Record/004232056 Sampling / Steven K. Thompson.]
+*[http://en.wikipedia.org/wiki/Stream_(computing)  Stream (Computing) Wikipedia]
-*[http://mirlyn.lib.umich.edu/Record/004133572 Sampling theory and methods / S. Sampath.]
+*[http://en.wikipedia.org/wiki/Cloud_storage  Cloud Storage Wikipedia]
-*[http://ori.hhs.gov/education/products/n_illinois_u/datamanagement/dhtopic.html Data Handling ]
+*[http://en.wikipedia.org/wiki/Data_warehouse  Data Warehouse Wikipedia]

@@ Line 78: / Line 78: @@
 &  3 &  1\\
        \end{bmatrix}$
 Example: R arrays and data-frames
@@ Line 93: / Line 90: @@
 3 l 2004-01-15
-data.matrix(DF[1:2])
+ data.matrix(DF[1:2])
-data.matrix(DF)
+ data.matrix(DF)
-−
   > data.matrix(DF)
       a.b     c
@@ Line 101: / Line 98: @@
 [2,] 2 2 12425
 [3,] 3 3 12432
-−
-> sleep 		# sleep dataset
+ > sleep 		# sleep dataset
   ......extra group ID
     0.7     1  1
@@ Line 124: / Line 121: @@
    4.6     2  9
    3.4     2 10
 '''Binary ASCII:'''  ASCII, which is short for American Standard Code for Information Interchange is a character-encoding scheme originally based on English alphabet that encodes 128 specified characters – the numbers 0 – 9, the letters a – z and A – Z, some basic punctuation symbols, some control codes that originated with teletype machines and a bland space – into the 7-bit binary integers. It represents text in computers, communications equipment, and other devices that use text. To convert the ASCII code to binary character (part):
@@ Line 144: / Line 139: @@
 |}
 </center>
 <center>
@@ Line 186: / Line 180: @@
 |}
 </center>
 '''Handling:''' the process of ensuring that the research data is stored, archived or disposed off in a safe and secure manner during and after the conclusion of a research project. This includes the development of policies and procedures to manage data handled electronically as well as through non-electronic means.
@@ Line 218: / Line 211: @@
   Static Mixture of Gaussians Data Stream
   With 3 clusters in 3 dimensions
-−
   > p <- get_points(dsd, n=5)
-−
   > p
           V1        V2        V3
-:1 0.6930698 0.4082633 0.3857444
+0.6930698 0.4082633 0.3857444
-:2 0.9086645 0.5545718 0.6942871
+0.9086645 0.5545718 0.6942871
-:3 0.6031121 0.5288573 0.5795389
+0.6031121 0.5288573 0.5795389
-:4 0.7590460 0.5485160 0.6484480
+0.7590460 0.5485160 0.6484480
-:5 0.7249682 0.7327056 0.4891291
+0.7249682 0.7327056 0.4891291
   > p <- get_points(dsd, n=100, assignment=TRUE)
-−
   > attr(p, "assignment")
-  : [1]  3  3  3 NA  2  2  2  3  1  2  1  2  2  3  1  1  1  3  3  2  3  1  1  2
+  [1]  3  3  3 NA  2  2  2  3  1  2  1  2  2  3  1  1  1  3  3  2  3  1  1  2
-  :[25]  3  1  3  3  1  1  3  2  3  3  2 NA  1  1  1  1  1  3  2  2  2  3  1  2
+  [25]  3  1  3  3  1  1  3  2  3  3  2 NA  1  1  1  1  1  3  2  2  2  3  1  2
-  :[49]  2  2  2  3  1 NA  1  3  2  2  3  3  3  2  1  2  2  2  1  3  1  1  1  2
+  [49]  2  2  2  3  1 NA  1  3  2  2  3  3  3  2  1  2  2  2  1  3  1  1  1  2
-  :[73]  1  3  2  2  3  1 NA  1  2  1  1  3  3  1  1  2  1  3  3  3  2  3  3  2
+  [73]  1  3  2  2  3  1 NA  1  2  1  1  3  3  1  1  2  1  3  3  3  2  3  3  2
-  :[97] NA  1  2  1
+  [97] NA  1  2  1
 Example 2: Import of big data: zips (the data would be available at http://media.mongodb.org/zips.json)  import using R/MongoDB:
 *Steps. Install the package of ‘rmongodb’ and import data zips, this data is included in the rmongodb package and can be loaded using the command data(zips):
@@ Line 250: / Line 243: @@
 ''Output:''
+ > install.packages('rmongodb')
-> install.packages('rmongodb')
+ trying URL 'http://cran.mtu.edu/bin/macosx/leopard/contrib/2.15/rmongodb_1.6.5.tgz'
+ Content type 'application/x-gzip' length 1291831 bytes (1.2 Mb)
-trying URL 'http://cran.mtu.edu/bin/macosx/leopard/contrib/2.15/rmongodb_1.6.5.tgz'
+ opened URL
-Content type 'application/x-gzip' length 1291831 bytes (1.2 Mb)
+ downloaded 1.2 Mb
 opened URL
-==================================================
+ The downloaded binary packages are in
-downloaded 1.2 Mb
+ /var/folders/k6/3r5dstw5385_b4fzt679pmsr0000gn/T//RtmpWD2wiV/downloaded_packages
-−
-The downloaded binary packages are in
+ > head(zips)
-*/var/folders/k6/3r5dstw5385_b4fzt679pmsr0000gn/T//RtmpWD2wiV/downloaded_packages
+ city               loc        pop       state _id
-−
-> head(zips)
+ [1,] "ACMAR"      Numeric,2 6055  "AL"  "35004"
+ [2,] "ADAMSVILLE" Numeric,2 10616 "AL"  "35005"
-city               loc        pop       state _id
+ [3,] "ADGER"      Numeric,2 3205  "AL"  "35006"
+ [4,] "KEYSTONE"   Numeric,2 14218 "AL"  "35007"
-:[1,] "ACMAR"      Numeric,2 6055  "AL"  "35004"
+ [5,] "NEW SITE"   Numeric,2 19942 "AL"  "35010"
-:[2,] "ADAMSVILLE" Numeric,2 10616 "AL"  "35005"
+ [6,] "ALPINE"     Numeric,2 3062  "AL"  "35014"
 ===References===

@@ Line 73: / Line 73: @@
 Types of arrays:
 *One-dimensional array: a simple example: an array of 8 integer variables with indices 0 through 7 may be sored as 8 words at memory address: {200, 202, 204, 206, 208, 210, 212, 214} which can be memorized as 200+2i.
-*Multidimensional arrays: Data=[(2&3&0@6&4&5@5&3&1)].
+*Multidimensional arrays: $Data=\begin{bmatrix}
 Example: R arrays and data-frames

@@ Line 66: / Line 66: @@
 </center>
-A well-known example would be the pipeline, which utilizes data from the LONI Image Data Archive (IDA). The pipeline takes advantage of the cluster nodes to download files in parallel from the IDA database. (http://pipeline.loni.usc.edu/learn/user-guide/building-a-workflow/#IDA)
+A well-known example would be the pipeline, which utilizes data from the LONI Image Data Archive (IDA). The pipeline takes advantage of the cluster nodes to download files in parallel from the IDA database.
-This article (https://ida.loni.usc.edu/login.jsp) provides a specific introduction to The LONI Image Data Archive (IDA), which offers an integrated environment for safely archiving, querying, visualizing and sharing neuroimaging data. It facilitates the de-identification and pooling of data from multiple institutions, protecting data from unauthorized access while providing the ability to share data among collaborative investigators. The archive provides flexibility in establishing project metadata, accommodating one or more research groups, sits and others. It’s simple, secure and requires nothing more than a computer with Internet connection and a web browser.
+*[http://pipeline.loni.usc.edu/learn/user-guide/building-a-workflow/#IDA Building a Workflow User-Guide]
 '''Arrays:''' a data structure consisting of a collection of elements, each identified by at least one array index or key.