Interpreting a multivariate analysis of functional neuroimaging data

A commentary on

Working memory circuitry in schizophrenia shows widespread cortical inefficiency and compensation
by Kim, M. A., Tura, E., Potkin, S. G., Fallon, J. H., Manoach, D. S., Calhoun, V. D., and Turner, J. A. (2010). Schizophr Res. 117, 42–51.

Over a decade ago, Nestor et al. (2002) employed a data-driven multivariate statistical algorithm to better understand brain-behavior correlates in schizophrenia. The technique used, partial least squares (PLS), is similar to principal component analysis and is able to uncover functional neural networks that covary with behavior ( McIntosh et al., 1996 ). Given that prominent theories posit schizophrenia to be a disorder stemming from dysfunctional connectivity (e. g., Friston and Frith, 1995 ), it is laudable to employ these advanced analysis techniques to optimally decipher functional neuroimaging data. However, we caution users that interpretation is prone to be misconstrued, especially given the familiarity of most users with univariate general linear model based analyses which offers somewhat different information, but, at the same time, underscores the complementary nature of these multivariate techniques.

A recent study employed PLS in order to indentify functional activation patterns from functional MRI (fMRI) data that covaried with their behavior measure, working memory performance, and to further identify how these patterns differ in patients with schizophrenia versus healthy comparison subjects ( Kim et al., 2010 ). The group noted that the brain scores for the healthy control sample were stable irrespective of task complexity, however the schizophrenia patients showed larger brain scores for the intermediate memory load condition. They further showed that for schizophrenia patients only, brain scores correlated with reaction times during the task. Based on these findings, the group concluded that they found evidence for “ concurrent hypoactivations and hyperactivations of various brain regions during a working memory task” in schizophrenia (p. 46; Kim et al., 2010 ). We suggest that the interpretation of these results oversteps what is actually reflected in the analysis. Kim et al. (2010) interpret their results as “ an exaggerated change in the BOLD signal” or “ increased activation” in schizophrenia patients relative to healthy controls. These claims are based on a greater magnitude of brain scores , which essentially represents the degree to which each behavior contributes to the covariance pattern of functional activation (see below). Those behaviors with higher brain scores are said to modulate or contribute to a greater extent to the functional neural network than behaviors with lower brain scores. Importantly, the brain score does not reveal any indication of increased or decreased activation per se . Such information can, however, be obtained by simply examining the individual hemodynamic response function (HRF) at a given voxel (for example see Diaconescu et al., 2011 ).

The aim of this commentary is not to highlight the technicalities of PLS, which has been done elsewhere ( McIntosh et al., 1996 ; McIntosh and Lobaugh, 2004 ); however, it is necessary to present a brief conceptual overview. PLS identifies patterns of brain voxels whose signal change covaries with experimental conditions or behavior across subjects. Task PLS takes the experimental conditions and signals at each voxel and calculates a covariance matrix, which is calculated across subjects. This covariance matrix then undergoes singular value decomposition that produces orthogonal latent variables (LVs) or singular values that optimally represent the relationship between brain activity and behavior. These LVs are extracted in the order of the amount of covariance explained between neural signal and behavior. The degree to which each subject expresses the behavioral pattern determined in the LV is defined as the brain score, which is calculated by multiplying the singular value vector by the raw image. Significance for each LV is determined using a permutation test that samples without replacement to reorder conditions and subjects. The proportion of times the original singular value is exceeded in the permutation test is used as an exact probability. Importantly, since the singular value decomposition is done simultaneously, corrections for multiple comparisons are not required. Next, to determine the reliability of the weights for the brain voxels that showed a pattern of behavior identified by the LV, standard errors are submitted to a bootstrap estimation that randomly samples with replacement. The ratio of weights to the bootstrap standard error is approximately equivalent to a z score, and can be used to threshold results to a conventional statistical standard (i. e., z score > x corresponds to p < y ).

Kim et al. (2010) report increased activation based on brain scores, which is an incorrect interpretation of their findings. It may be the case that they did examine the individual HRFs and these serendipitously happened to be in the same direction of the brain scores, and thus their claims are correct. However, due to the direct references to brain scores in their paper, we suspect that this is not the case. Their results might better be interpreted as showing greater modulation of the emergent networks in high-demand conditions compared to low-demand conditions in schizophrenia patients relative to controls.

Summarizing, we wish to affirm the utility of advanced multivariate statistical algorithms for the analysis of behavior and functional neuroimaging data together, but also stress the importance of proper interpretation of the complex results. Employing such techniques, especially in concert with traditional univariate analyses, may prove to be an effective analysis strategy to uncover the complex relationship between behavior and neural activity in mental illness. Some researchers have done just this and produced fruitful results ( Koutsouleris et al., 2010 ), which underscores the promise of these techniques.

References

Diaconescu, A. O., Jensen, J., Wang, H., Willeit, M., Menon, M., Kapur, S., and McIntosh, A. R. (2011). Aberrant effective connectivity in schizophrenia patients during appetitive conditioning. Front. Hum. Neurosci. 4: 239. doi: 10. 3389/fnhum. 2010. 00239

Friston, K. J., and Frith, C. D. (1995). Schizophrenia: a disconnection syndrome? Clin. Neurosci. 3, 89–97.

Kim, M. A., Tura, E., Potkin, S. G., Fallon, J. H., Manoach, D. S., Calhoun, V. D., and Turner, J. A. (2010). Working memory circuitry in schizophrenia shows widespread cortical inefficiency and compensation. Schizophr. Res. 117, 42–51.

Koutsouleris, N., Gaser, C., Bottlender, R., Davatzikos, C., Decker, P., Jager, M., Schmitt, G., Reiser, M., Moller, H. J., and Meisenzahl, E. M. (2010). Use of neuroanatomical pattern regression to predict the structural brain dynamics of vulnerability and transition to psychosis. Schizophr. Res. 123, 175–187.

McIntosh, A. R., Bookstein, F. L., Haxby, J. V., and Grady, C. L. (1996). Spatial pattern analysis of functional brain images using partial least squares. Neuroimage 3, 143–157.

McIntosh, A. R., and Lobaugh, N. J. (2004). Partial least squares analysis of neuroimaging data: applications and advances. Neuroimage 23(Suppl. 1), S250–S263.

Nestor, P. G., O’Donnell, B. F., McCarley, R. W., Niznikiewicz, M., Barnard, J., Jen Shen, Z., Bookstein, F. L., and Shenton, M. E. (2002). A new statistical method for testing hypotheses of neuropsychological/MRI relationships in schizophrenia: partial least squares analysis. Schizophr. Res. 53, 57–66.