Multi-Voxel Pattern Analysis (MVPA) For FMRI Principles .
MVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMulti-Voxel Pattern Analysis (MVPA) forfMRIPrinciples and Methods7th M-BIC fMRI Workshop 2012MVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsGiancarlo ValenteMaastricht University, Department of Cognitive NeuroscienceMaastricht Brain Imaging Center (M-Bic), Maastricht, The Netherlands22 March 2012
OverviewMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results1 IntroductionMultivariate approaches in fMRIClassificationRegression2 Machine learning for fMRI: PrinciplesFrameworkLearning a model3 MVPA: HOWTOPre-Processing and Feature ExtractionFeature SelectionModel training and testingStatistical assessment of MVPA results
Univariate Statistics for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteConventional (voxel-by-voxel) statisticsFunctional imagesIntroduction 2sMultivariateapproaches ignal(% change)Machine LearningFrameworkLearning amodelTimeMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsConditionStatistical Mapsuperimposed onanatomical MRI imageTimeVoxel/Region of interest (ROI) 5 min
Multivariate models for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results
Multivariate models for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDoes theactivity in the two voxels covary?Functional connectivity
Multivariate models for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionDoes theactivity in the two voxels covary?Functional connectivityMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDoes the activity in one voxelinfluence the activity in the other?Effective connectivity
Multivariate models for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionDoes theactivity in the two voxels covary?Functional connectivityMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDoes the activity in one voxelinfluence the activity in the other?Effective connectivityDo they jointly convey informationon a stimulus or a brain state?Multi-voxel pattern analysis(MVPA), ‘brain reading’
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsTo deal with multiple independent variables and multiple dependentvariables, it is possible to employ multivariate statistical tests, e.g.Multivariate Analysis of Variance (MANOVA) or Canonical VariateAnalysis (CVA).Each independent variable takes up a degree of freedom, therefore incase of large amount of voxels, it is troublesome to employ them.
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsTo deal with multiple independent variables and multiple dependentvariables, it is possible to employ multivariate statistical tests, e.g.Multivariate Analysis of Variance (MANOVA) or Canonical VariateAnalysis (CVA).Each independent variable takes up a degree of freedom, therefore incase of large amount of voxels, it is troublesome to employ them.Reduce the amount of voxels using a searchlight approach,[Kriegeskorte06])Reduce the amount of voxels using PCA, PLS, MLM,[Friston95a, McIntosh96, Worsley97]Test for a multivariate effect in a different way. (MVPA)
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsDistributed representation of faces and objects in ventral temporalcortex [Haxby01]Giancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionObject photos presented incategory blocksMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsHaxby et al. (2001)
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsCorrelation analysisof spatial response patternsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionL RMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsHaxby et al. (2001)
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionPattern comparison by categorypairMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsHaxby et al. (2001)
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariate analysis can handle Multivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsweak-at-every-voxeldistributed effectsopposite selectivityin adjacent voxels
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsPatterns as points
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsAn Aid to Intuition about Multivariate Analysis: Part 1Giancarlo ValenteIntroductionIdeal Univariate DataMultivariateapproaches infMRIClassificationRegressionCondition ACondition BBaselineMachine LearningFrameworkLearning amodelMVPA: HOWTOVoxel 2ActivityPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDavid Cox and Robert SavoyActivity
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsAn Aid to Intuition about Multivariate Analysis: Part 2Giancarlo ValenteIntroductionLinearly Separable, but Problematic in Univariate AnalysesMultivariateapproaches infMRIClassificationRegressionCondition ACondition BBaselineMachine LearningFrameworkLearning amodelVoxel 2MVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsActivityDavid Cox and Robert SavoyActivity
Multi-Voxel Pattern AnalysisMVPA for fMRI Principles andMethodsAn Aid to Intuition about Multivariate Analysis: Part 3Giancarlo ValenteIntroductionNonlinearly Separable DataMultivariateapproaches infMRIClassificationRegressionCondition ACondition BBaselineMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsActivityVoxel 2ActivityVoxel 1 ActivityDavid Cox and Robert SavoyVoxel 1Voxel 2
Classification: an example from [Cox03]MVPA for fMRI Principles andMethodsBasic Format of the Experiments and Analysis: Part 1Giancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegression“Brain Reading”: Classifying experience using patterns of neural activityTRAININGMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsclassifierlabel: “basket”voxelsDavid Cox and Robert Savoy
Classification: an example from [Cox03]MVPA for fMRI Principles andMethodsBasic Format of the Experiments and Analysis: Part 2Giancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionCLASSIFICATION of data from a subsequent sessionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsTrained classifierbest guess:“basket”voxelsDavid Cox and Robert Savoy
Classification: an example from [Cox03]MVPA for fMRI Principles andMethodsSelection of StimuliGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionStimuliWe have begun by using complex, real world, visual objects, from 10different groups, presented as gray-scale images.Machine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDavid Cox and Robert Savoy
Classification: an example from [Cox03]MVPA for fMRI Principles andMethodsComparing 10 classifiers (from one subject)Giancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionNo one voxel or small subset of voxels is significantlydifferent between category for single trialsMachine LearningMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsnorm. MR signalFrameworkLearning amodel200-2050100voxel numberDavid Cox and Robert Savoy150
Classification: an example from [Cox03]MVPA for fMRI Principles andMethodsEffect of Voxel Selection Set SizeGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsDavid Cox and Robert Savoy
Regression example: PBAIC 2007 competitionMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionPBAIC 2007: Pittsburgh Brain Activity Interpretation CompetitionInterpreting subject-driven actions and sensory experience in arigorously characterized virtual world,organized by Walter Schneiderand Greg Siegle of the University of Pittsburgh,www.braincompetition.org.Machine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsFigure 1: c 2007 University of Pittsburgh
Regression example: PBAIC 2007 competitionMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegression3 subject operating in a virtual reality worldNovel Virtual Reality world task: searching for and collectingobjects, interpreting changing instructions, and avoiding athreatening dogLearn from Run1 and Run2 and in Run 3 predict feature vectorsfrom brain activity dataMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsFigure 2: c 2007 University of Pittsburgh
Regression example: PBAIC 2007 competitionMVPA for fMRI Principles andMethodsGiancarlo ValenteUse the data from Run1 and Run2 to develop the ability to go fromthe brain activation data to the feature dataIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsFigure 3: c 2007 University of Pittsburgh
Regression example: PBAIC 2007 competitionMVPA for fMRI Principles andMethodsPredict unknown ratings from fMRI data of Run3Giancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsFigure 4: c 2007 University of Pittsburgh
Regression example: PBAIC 2007 competitionMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsUsing Relevance Vector Machine [Tipping01, Valente11]
OverviewMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results1 IntroductionMultivariate approaches in fMRIClassificationRegression2 Machine learning for fMRI: PrinciplesFrameworkLearning a model3 MVPA: HOWTOPre-Processing and Feature ExtractionFeature SelectionModel training and testingStatistical assessment of MVPA results
Machine Learning for fMRIMVPA for fMRI Principles andMethodsConsider the following datasets:Giancarlo ValenteDataset1: Data X and labels t (Training dataset)IntroductionDataset2: Data X’ and labels t0 (Test dataset, not availableduring training!)Multivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultst can beDiscrete: Classification (or ordinal regression)Continuous: RegressionWith machine learning we aim at finding, on the training data X andt, a suitable functionf f (X, θ)where θ denotes a set of model parameters, such thatt̃ f (X0 , θ)is a “good” estimate of t0
Machine Learning for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsThe learnt model f should be such that it minimizes a measure oferror on the test dataset (generalization).The most employed measures are:Classification: Classification accuracy, Area under the CurveRegression: Correlation, Root Mean Square Error (RMSE)
Machine Learning for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelThe learnt model f should be such that it minimizes a measure oferror on the test dataset (generalization).The most employed measures are:Classification: Classification accuracy, Area under the CurveRegression: Correlation, Root Mean Square Error (RMSE)MVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsHow do we learn the model?
Machine Learning for fMRIMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelThe learnt model f should be such that it minimizes a measure oferror on the test dataset (generalization).The most employed measures are:Classification: Classification accuracy, Area under the CurveRegression: Correlation, Root Mean Square Error (RMSE)MVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsHow do we learn the model?Minimizing the error on the training data?
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsConsidera simple 1-D example(from [Bishop06]).To “learn” a functionalrelationship (ideal:GREEN) on training data(RED) and generalizeon test data (BLUE), weuse a polynomial model:y a0 a1 x a2 x2 . . . an xn1.5Real modelTRAINING dataTESTING data10.50-0.5-1-1.500.20.40.60.81
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionWe optimize the polynomial coefficients by minimizing theleast-square error (Maximum Likelihood (ML) solution, if the noisehas a Gaussian distribution)Machine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results(a) order 2(b) order 3(c) order 5
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionThe RMS error decreases with the polynomial order increasingMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results(d) order 7(e) order 8(f) order 10
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionExamining the performances on the test dataset:Multivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results(a) order 2(b) order 3(c) order 5
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionA better fit on the training dataset does not always imply a better fiton the test datasetMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA results(d) order 7(e) order 8(f) order 10
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMachine LearningFrameworkLearning amodelMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsWitha high order polynomial we fitperfectly the training data, buthave higher error on test data.In this case training datahave been OVERFITTED:we have considerednoise as interesting signal3Error on Training DatasetError on Testing Dataset2.521.510.5012345678910
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches e sizeESTIMATION PROCEDUREMachine LearningFrameworkLearning amodelOverfittingMVPA: HOWTOPre-Processingand FeatureExtractionFeatureSelectionModel trainingand testingStatisticalassessment ofMVPA resultsGiven the available samples, our model was too complex. Asimpler model would have better performancesif more samples were available, there will be less overfitting withthe same polynomial order.The estimation procedure (Least Square) tends to favoroverfitting of the training data.
Learning a modelMVPA for fMRI Principles andMethodsGiancarlo ValenteIntroductionMultivariateapproaches infMRIClassificationRegressionMa
fMRI Principles and Methods 7th M-BIC fMRI Workshop 2012 Giancarlo Valente Maastricht University, Department of Cognitive Neuroscience Maastricht Brain Imaging Center (M-Bic), Maastricht, The Netherlands 22 March 2012. MVPA for fMRI - Principles and Methods
MRI Physics Anthony Wolbarst, Nathan Yanasak, R. Jason Stafford . Introduction to MRI ‘Quantum’ NMR and MRI in 0D Magnetization, m(x,t), in a Voxel Proton Density MRI in 1D T1 Spin-Relaxation in a Voxel MRI Case Study, and Caveat Sketch of the MRI Device ‘Classical’ NMR in a Voxel
-%&5 #''5&*% (#'7 2'1 !', -'', !6 Figure 5. 2D t-SNE embedding of the last layer features learned from the 3D ConvNet. Color encodes object category. spaced 3D voxel grid. The value in each voxel is deÞned to be the shortest distance between the voxel center and the surface from the input depth map. Figure 3 shows a few ex-amples.
Voxel cone tracing - low resolution, can’t represent walls used in architecture with affordable voxel sizes in practice Precomputed AO volumes - low resolution for self shadowing, difficulty avoiding over occlusion from multiple objects. . Visualization – Trace ca
For this purpose, the underlying image properties with regard to background and signal-to-noise ratio are extracted for each voxel / volume segment as described above and the optimal, voxel-accurate parameter sets are provided for the subsequent deconvolution on this basis. LIGHTNING White Paper, September 2018 6
indirect illumination in a bottom-up fashion. The resulting hierarchical octree structure is then used by their voxel cone tracing technique to produce high quality indirect illumination that supports both diffuse and high glossy indirect illumination. 3 Our Algorithm Our lighting system is a deferred renderer, which supports illumination by mul-
Set up 3D DDA (Digital Differential Analyzer) Similar to Bresenhm's line drawing algorithm Set up 3D DDA (Digital Differential Analyzer) blue values changes along the traversal NextCrossingT[1] Out voxel gg index DeltaT[0] rayT Step[0] 1 Pos NextCrossingT[0] DeltaT: the distance change when voxel changes 1 in that direction Set up 3D DDA
Set up 3D DDA (Digital Differential Analyzer) blue values changes along the traversal NextCrossingT[1] Out voxel gg index DeltaT[0] rayT Step[0] 1 Pos NextCrossingT[0] DeltaT: the distance change when voxel changes 1 in that direction. Set up 3D DDA for (int axis 0; axis 3; axis)
Image 34 Charles Duncan – A Modern Approach to Classical Guitar 106 Image 35 Mario Rodriguez Arenas – The School of the Guitar 108 Image 36 Julio Sagreras – First Guitar Lessons 109
