A Tutorial On Data Analysis

Classification of samples Cluster analysis vs. classification Use cluster analysis to discover new classes, or forvisualization purposes Use classification when classes are already specified Classification is supervised learning, and generally hasmore power because it uses the known information aboutthe hybridized samples. Use the Class Prediction tool when the primary interest isto form a classifier to predict the class of new samples.38

Class comparison toolBetween groups of arrays FOR SINGLE-CHANNEL DATA, OR DUALCHANNEL REFERENCE DESIGNS. Class comparison tool uses univariate t/F-tests,with multivariate permutation tests Permutation tests are nonparametric, and takecorrelation among genes into account Paired analysis option Produces a gene list which can be used for furtheranalysis. Produces chromosomal distribution and GOanalysis if genes have already been annotatedusing the Affymetrix or SOURCE database.39

Class comparison toolBetween red and green channels Similar to the other class comparison tool, but comparesbetween red and green samples within arrays, rather thanbetween the red samples on groups of arrays. Used for Block Design, or for a Reference Design wherethe comparison of interest is between the test samples andreference sample rather than between test samples. Example of situation where this tool might be used:– Array1 contains (red test1, green control1)– Array2 contains (red test2, green control2)– Array3 contains (red control1, green test1), etc. Experiment stControlGreenClassControlControlTest40

Class comparison toolBlocking FactorExperimental designs containing a blocking factor can beperformed by specifying which column in the Experimentdescriptor worksheet contains a blocking variable. Whenselected, the influence of the blocking variable is taken intoconsideration when analyzing the differences between classes.Examples of variables that may be considered as Blockingfactors:¾ Clinical Site for patient data¾ Print set for cDNA spotted arrays¾ Batch of arrays41

Class comparison toolRandom variance option The random variance test has more powerbecause the “average” variance in thedenominator adds degrees of freedom forthe test statistic. Should be used for small sample sizes.42

Class comparison toolUnivariate significance test Compute the univariate p-value for each gene, andsort list of genes by smallest p-value. In the univariate setting (i.e., testing significanceof one gene at a time), the p-value is defined to bethe probability of obtaining a false positive result. However, once a list of univariately significantgenes is found, it is not clear how many of thosegenes are false positives.43

Class comparison toolMultivariate permutation test In the multivariate setting (i.e., when testing many genesfor significance at the same time), ask the question: Whatp-value cutoff should I use to guarantee that 90% of thetime, I get less than P proportion of false positives (whereP is specified by the user)? To answer this question, we compute the permutationdistribution of the p-value cutoffs for which we would getP proportion of false positives. The output tells us how far down the list we would be ableto go in order to be assured (with a certain confidence) ofgetting less than P proportion of false positives.44

[Hands-on instructions][Class comparison – Restricting proportion of false positives]1.2.3.4.5.Using the Pomeroy data, run the Class Comparison tool byclicking on ArrayTools Æ Class comparison Æ Betweengroups of arrays.Select the Dx variable as the column defining the classes.Select the Random variance model option, and select theRestriction on proportion of false discoveries with maximumproportion 0.1 and 90% Confidence level.Leave all other options at default levels. Now click OK on themain dialog to launch the analysis.You will see a DOS window appear in your Windows Task Barat the bottom of your screen. If you click on the DOS window,you can monitor the analysis running inside the DOS window.When the analysis has completed, it will automatically open up45an HTML file which displays the output.

Class comparisonSignificance Analysis of Microarrays (SAM) SAM is another popular method for falsediscovery control, which controls the averageproportion of false discoveries rather than theprobability of a given number or proportion offalse discoveries. It is a slightly less stringent control than themultivariate permutation test for controlling falsediscoveries used in the other class comparisontools, but is included in BRB-ArrayTools becauseof its popularity.46

[Hands-on instructions][Significance Analysis of Microarrays – Pomeroy data]1.2.Still using the Pomeroy data, run the SAM tool by clicking onArrayTools Æ Class comparison Æ Significance Analysis ofMicroarrays (SAM).Again, select the Dx variable as the column defining the classes,select the 90th percentile option, and leave all other parametersat default levels. Now click OK to exit the options dialog, andclick OK on the main dialog to launch the analysis.47

Gene ontology analysis In the class comparison, class prediction, survivalanalysis, or quantitative traits analysis output, theobserved vs. expected frequency is computed foreach Gene Ontology class represented in theselected genelist, as well as for each upstreamGene Ontology class. By default, results areprinted only for classes represented by at least fivegenes in the selected genelist, and with anobserved versus expected ratio of at least 2.48

Gene set Expression Comparison Allows users to find significant sets of genes rather than justsignificant genes.For the Gene Ontology comparison, all Gene Ontology classes thatare represented in the data are tested for significance.For Pathway Comparison, all the pathways that are represented in thedata are tested. For Human, the BioCarta or KEGG pathways aretested and for mouse, the BioCarta pathways are compared.Additionally, Broad/MIT pathways can be downloaded to be used inanalyses.For the User Gene Lists comparison, the user can select specificgenelists that the user would like to test for significance.For each group of gene that is tested for significance, a summarymeasure of the univariate p-values of the genes within that group iscompared against summary measures obtained from a randomsampling of groups of genes of the same size. A permutation p-valuefor the group of genes is obtained by ranking its actual p-valuecompared to the random sampling.49

Gene Set Expression Comparison Compute p-value of differential expression foreach gene in the gene set(k number of genes) Compute a summary (S) of these p-values Determine whether the summary test (S) is moreextreme than would be expected from a randomsample of “k” genes on that platform. Two types of summaries provided:- Average of log p-values- Kolmogrov-Smirnov statistic.50

Hotelling’s T-square test Tests the hypothesis that no genes in the gene setare significant. Principal components are computed for all thegenes in the gene set using all the arrays. Hotelling’s T-sq is a multivariate test of theunivariate t- and f- tests for testing the classmeans. Uses a large sample approximation (min # ofarrays 20)51

[Hands-on instructions][Class Comparison – Pathway Comparison: Pomeroy data]1.On the Pomeroy data, run the Class Comparison tool by clickingon ArrayTools Æ Class comparison Æ Gene set ExpressionComparison.2. Select the Dx variable as the column defining the classes. Selectthe Random variance model option and Pathways, and leave allother options at default levels. Now click OK on the main dialogto launch the analysis.3. You will see a DOS window appear in your Windows Task Bar atthe bottom of your screen. If you click on the DOS window, youcan monitor the analysis running inside the DOS window.4. When the analysis has completed, it will automatically open up an52HTML file which displays the output.

Quantitative trait tool Selects genes which are univariately correlatedwith a quantitative trait such as age or time point. Controls number and proportion of falsediscoveries in entire list: uses a multivariatepermutation test which takes advantage of thecorrelation among genes. Produces a gene list which can be used for furtheranalysis. Produces chromosomal distribution and GOanalysis if genes have already been annotatedusing the SOURCE database.53

Survival analysis tools Find Genes Correlated with Survival tool, selects geneswhich are univariately correlated with survival Controls number and proportion of false discoveries inentire list: uses a multivariate permutation test which takesadvantage of the correlation among genes Produces a gene list which can be used for further analysis. Produces chromosomal distribution and GO analysis ifgenes have already been annotated using the SOURCEdatabase.54

Exercise-IUsing the Perou sample data set, find genes that aredifferentially expressed for patients before and aftertreatment :? Obtain a gene list that contain no more than 40% ofFalse discoveries with 95% confidence.? Choosing an alternative method to control for falsediscoveries obtain another genelist with a 95%confidence level and controlling for 40% Falsediscoveries.? Explain the differences between the two outputs.? Using the common set of significant genes obtainedfrom the above two analyses, run a scatterplot ofphenotype averages with 2 fold difference andcomment on the downward regulated genes.55

Components of Class Prediction Feature(gene) selection-which genes will be included in the model. Select model type.-choose prediction method (DLDA,CCP etc) Fit the parameters for the model. Evaluating the Classifier- Cross-validation56

Class prediction tool Six methods of prediction:Compound covariate predictor (2 classes only)Bayesian Compound covariate predictor (2 classes only)K-nearest neighbor (2 or more classes)Nearest centroid (2 or more classes)Support vector machines (2 classes only)Diagonal linear discriminant analysis (2 or more classes) Selection of genes may be based on univariate significancecriterion or univariate misclassification rate, and minimumfold-ratio of geometric means. The univariatemisclassification rate criterion is available when there areonly two classes. In addition,we have added the option to select genes using“gene pairs”by the “greedy pair”method –Bo & Jonassen57

Cross-validating the classifier Leave-One-Out cross validation.K-Fold cross validation. 0.632 bootstrap cross-validation.Use leave-one-out cross-validation to compute amisclassification rate Re-compute the classifier, based on all but onesample Use the classifier to classify the sample which hasbeen left out58

Permutation test Use a permutation test to assess the significance ofthe misclassification rate and univariatesignificance of each gene For each permutation of the class labels, re-run thecross-validation and obtain a new cross-validatedmisclassification rate The permutation p-value is based upon the rank ofthe misclassification rate using the original data,compared to all permutations59

Compound covariate predictor May only be used for classifying among two classlabels Select genes which univariately classify thesamples Form a compound covariate predictor as:Σitixi{where ti t-statistic, xi log-ratio,and sum is taken over all significant genes Determine the cutpoint of the predictor as themidpoint between its mean in one class and itsmean in the other class60

Diagonal linear discriminant analysis May be used for classifying among two or moreclass labels Use F-test to screen for genes which areunivariately significant in classifying the samples Seeks a linear combination of the variables whichhas a maximal ratio of the separation of the classmeans to the within-class variance, where genesare assumed to be uncorrelated61

Bayesian Compound Covariate Compound Covariate score is computed forall the samples in the cross-validatedtraining set. The CCP-scores of samples in each class ofthe training set are assumed to be from aGaussian distribution. If prior probabilities are ½ - the BCCP issimilar to the CCP.62

K-nearest neighbor May be used for classifying among two or moreclass labels Use F-test to screen for genes which areunivariately significant in classifying the samples For k 1 and k 3, finds the k-nearest neighbors interms of Euclidean distance over only those geneswhich were univariately significant Classify based on the majority vote of the classlabels of the k-nearest neighbors63

Nearest centroid May be used for classifying among two or moreclass labels Use F-test to screen for genes which areunivariately significant in classifying the samples Compute the centroid of each class as a mean overall the training samples with that class label Classify test sample to be same class label as thenearest centroid, using Euclidean distance overonly those genes which were univariatelysignificant64

Support vector machines(V. Vapnik) Implemented only for classifying among two classlabels Select genes which univariately classify thesamples The SVM predictor is implemented as a linearfunction of the log-ratios or the log-intensitiesover the significant genes, that best separates thedata subject to penalty costs on the number ofspecimens misclassified.65

Class prediction toolClass prediction vs. binary tree prediction The class prediction tool has more options: may select allprediction methods simultaneously, may use pairedsamples, may use randomized variance option. The binary tree prediction tool splits the classes intogroups of subclasses. At each node in the tree, the binarytree prediction tool decides how to split the classes intotwo groups based on either a leave-one-out or a K-foldcross-validation. The binary tree prediction tool may beuseful if there is a hierarchical structure to the classes. However, the binary tree prediction may be very slow for alarge number of samples. Therefore, a K-fold crossvalidation should be used if the number of samples islarge. Currently the tool is limited to five classes, and requires atleast four samples per class for good prediction.66

Prediction Analysis MicroarrayPAM Uses Shrunken Centroid algorithm developed byTibshirani’s group (Stanford). Similar to Nearest Centroid but the centroids areshrunk towards each other based on shrinking theclass means for each gene towards an overallmean. Amount of shrinking is determined by a tuningparameter delta and the number of genes includedin the classifier is determined by the value ofdelta.67

[Hands-on instructions ][Class prediction –Pomeroy data]1.2.3.4.5.6.7.Run the Class Prediction tool by clicking on ArrayTools ÆClass prediction Æ Class prediction.Select the Dx variable as the column defining the classes. Checkthe box for using the Random Variance Model.Choose the univariate significance alpha 0.001.De select the Compound Covariate Predictor,BCCP and SVM.Select Options, check the box for Use separate test set, andselect the column “TrainingSet”.Leave all other options at default levels, and click OK.Note the Array Ids which have been misclassified by allmethods.68

Survival Risk Prediction Develops a gene expression based predictor ofsurvival risk groups allowing for up to 3covariates in the model. Selects genes that correlate with survival based ona p-value threshold. First k-principal components are computed. A k-variable Cox proportional hazards regressionis performed with k pc’s are predictors.69

Survival Risk Group Prediction Cross-validate using LOOCV or K-fold Develop supervised pc PH model for training set. Compute cross-validated predictive index for ithcase using the PH model for training set. Compute the predictive risk percentile index for ithcase among the predictive indices for the cases inthe training set. Plot K-M survival curves. Compute the log-rank statistic comparing thecross-validated K-M curves.70

[ Hands-on instructions ]Survival Risk Prediction Using the Pomeroy data set, clicking on ArrayTools ÆSurvival Analysis Æ Survival Risk prediction.Select “SurvStatusCode” for the survival status columnand “Survival(months)” as the survival time column.Keeping all other options as defaults. Select the “Age”as a clinical covariate.Now click OK on the main dialog to launch the analysis.71

Hierarchical clustering tools Clustering of genes and samples produces visual imageplot of log-expression data, where ordering is determinedby ordering of dendrogram Can compute measures to assess cluster reproducibilitywhen clustering samples alone May cluster based on gene subsets rather than on the entiregene set Interface to Cluster 3.0 and TreeView originally producedby the Stanford group is also included, and allows for easyexportation of results.72

[Hands-on instructions][Cluster analysis – Pomeroy data]1.2.3.4.Using the Pomeroy data set.Run the cluster analysis by clicking on ArrayTools ÆClustering Æ Genes (and samples).Click on the Select gene subsets button, and under Select genesfor analysis, choose the ClassComparison genelist, and clickOK.Now click on the Options button, and choose Dx as the variableunder Label the experiments. Click OK to exit the optionsdialog, and click OK on the main dialog to launch the analysis.73

[Hands-on instructions – cont’d][Cluster analysis – Pomeroy data]5.6.The analysis will open up a Cluster viewer worksheet insideyour project workbook. The first plot presented is the Heat Mapimage in a draft form. Using Zoom and Recolor button you canchange the color scheme of the map. Click the button and on thedialog page select Red/Blue scheme and de-select the Usequantile data This coloring option should look familiar to thedChip users.The setting for using the quantile data ranges when distributingcolors on the scale leads to the heat map when two differentmajor col

[Hands-on instructions] [Getting started] 1. Open Excel. 2. Click on Tools ÆAdd-ins, and see that BRB- ArrayTools is loaded as an add-in. 3. When BRB-ArrayTools is loaded as an add-in