Functional charlatans

Rinckside 2015; 26,4: 9-11.

his year, functional brain imaging celebrates its 25 anniversary. Basically, funct­ional imag­ing or fMRI is a misleading term because it is mainly used for the depiction of changes of local blood supply in the brain activated by specific stim­uli. In contrast to EEG and MEG, it does not provide a direct measure of neural activity [1].

   In 1990, Dr. Jack Belliveau and colleagues pub­lished the first observation of the stimulation of the human visual cortex by magnetic resonance imaging [2]. They watched the first pass effect of a contrast agent after bolus injection to demonstrate changes in cortical perfusion upon activation with a photic stimu­lus. However, this approach required two con­trast agent injections. This disadvantage was resolved by the demonstration of brain activation using the blood oxygenation level-dependent (BOLD) effect by Dr. Seiji Ogawa [3], also in 1990. Over the last few years, this very elegant technique has led to a fast proliferation of fMRI all over the world.

A real explosion of medical, paramedical, neuro-­social and neuro-economic publications ensued. The outcome of commercial applications went through the media and the yellow press: of the German auto­mobile manufacturer trying to find out which car de­sign attracts more men, of a major US soft drink company checking the most bewitching taste … you name it. fMRI is even used as a lie detector: No sci­ence, just fiction.

Soon, research in this topic was in the hands of ama­teurs playing with MR imaging and functional MR, lacking the background in physics, chemistry, and medicine – and the scientific rigor necessary to work in a new field. They saw pictures with colorful en­hancement of the brain and overnight became cogni­tive social neuroscientists, new-age phrenologists – because color pictures can be interpreted by every­body. But nobody bothered to ask: what is normal and what is pathological – or, what is an artifact?

   Unfortunately, BOLD studies have a very low sensitivity and signal-to-noise ratio. The signal changes related to cerebral activation are close to the noise level and therefore numerous signal processing and, beyond this, statistical techniques are used to overcome this handicap. Many blood flow alterations described in functional brain imaging rely on sig­nal-intensity changes of less than 5%. More so, T2* to estimate blood oxygen saturation is only one singled-out factor; oxygen supply and saturation are depen­dent on several additional and independent parame­ters, among them lung and heart function, vessel size, and hematocrit.

Even the inventor, Seiji Ogawa, added some very de­tailed and critical remarks about the techniques in a review article twenty-two years after the first de­scription of BOLD [4].

As Gustav von Schulthess pointed out in the early days of fMRI:

“... a caveat for fMRI: it is a very interesting tech­nique but signal changes are but a few percent. Hence, the method is technically demanding and ‘the threshold of nonsense production is low’ [5].”

An outstanding proof of his claim can be found read­ing one of the most famous publications in this field published in the last years: the fMRI story (at 1.5 Tesla) of a dead Atlantic Salmon (Salmo salar) [6]. Here are some excerpts.

From the Methods Section:

“The task administered to the salmon involved completing an open-ended men­talizing task. The salmon was shown a series of pho­tographs depicting human individuals in social situa­tions with a specified emotional valence, either so­cially inclusive or socially exclusive. The salmon was asked to determine which emotion the individual in the photo must have been experiencing. The photo stimuli were presented in a block design, ...”

The beginning of the Results Section:

“A t-contrast was used to test for regions with significant BOLD signal change during the presentation of photos as compared to rest. The parameters for this comparison were t(131) > 3.15, p(uncorrected) < 0.001, 3 voxel extent threshold. The relatively low extent threshold value was chosen due to the small size of the salmon’s brain relative to voxel size. Several active voxels were observed in a cluster located within the salmon’s brain cavity. The size of this cluster was 81 mm³ with a cluster-level significance of p = 0.001.”

This article presents in a really imaginative way the often overlooked main problem of fMRI. If the fMRI study of the little brain of a dead fish appears to give cognitive social scientists indications of brain func­tions and answers to some of their puzzles, how much confidence can we have in studies that follow the same or similar paradigms in far bigger live hu­man brains? With their very catching experiment and a later paper, Bennett and collaborators stressed how pivotal it is in fMRI to apply statistics properly and scrupulously because random noise may yield spuri­ous results in the acquired images [7].

   Some years ago I pointed out that the number of good medical – among them radiological – papers is less than one percent of all papers published. Much of the rest is without rhyme or reason, more chaff than grain. It seems even worse in fMRI. For that reason, good pa­pers are laudable – and who cares about bad papers, as long as they disappear in the sea of scientific trash? Still, the scientific reputation of fMRI research is poor, and the charlatans ruin the standing of the se­rious scientists in the field.

Sadly, a few papers are bad and ugly.

Sadly, a few papers are bad and ugly. During the last years I followed the publications of a group of au­thors that now should be marked as dubious, worry­ing, and malign. These people did something similar with presumed pedophiles as the others did with the dead salmon. They tried to identify pedophiles by changes of BOLD effects after showing the study subjects pictures, in this case of nude adults and chil­dren. For these experiments, the authors used small heterogeneous groups of presumed pedophiles at­tracted to either boys or girls, and similarly sized comparison groups of healthy heterosexual and ho­mosexual men. They found BOLD activation when applying (uncorrected) thresholds, and claim to be able to distinguish between pedophiles and non-pe­dophiles; their accuracy is 95% [8].

In a study published two years later about stimulation of what appears the same study subjects, this time shown pictures of adult and children faces, whole sections are poorly written and incomprehensible, the materials and methods section is irreproducible, the statistics are unsound and not applicable, and all re­sults are probably false positives [9]. More so, there are always the caveats of very small comparison groups and the lack of knowledge of what is normal.

The problem is not the bad research performed; this is common; it's the ugly and deeply disturbing con­clusion, where the authors turn dilettantism into a weapon:

“Functional brain response patterns to sexual stimuli contain sufficient information to identify pedophiles with high accuracy. The automatic classification of these patterns is a promising objective tool to clini­cally diagnose pedophilia.” [8]

This conclusion is, politely phrased, highly problem­atic. Using fMRI as a biomarker (i.e., a detector) for pedophilia is unethical – because the technique does not allow to identify pedophiles. The employment of fMRI to diagnose pedophilia may have unforeseen consequences. It is a misuse and abuse of medical imaging. None of these articles has the rock-solid foundation which would be necessary for the conclu­sions the authors draw at the end.

The impact of such papers might be hurtful and detri­mental, even deadly for some members of our soci­eties. Readers of the articles might draw conclusion and take actions that are not appropriate, taking for granted that “scientific” publications even in obscure journals can be taken as the last truth.

   The authors (Ponseti et al. from Kiel, Siebner from Copenhagen, and Beier et al. from Berlin) and the ethical committees of the respective universities as well as the editors and reviewers of the journals the articles were published in are responsible for pos­sible harm caused. The authors' claims are false and have to be actively countered as forcefully as possi­ble. The papers should be retracted.

Note: Writing this column has taken a good year. References 6, 8, and 9 have been retrospectively peer-reviewed by four leading scientists in the field. All of them thought that paper 6 is one of the best pa­pers ever written in fMRI; and all of them rejected papers 8 and 9. Meanwhile, some critique of the two latter papers (8 and 9) has appeared on the Web. It con­curs in what is said in this column. More so, state­ments that many neuroimaging papers are of in­ferior quality are also being underlined in numerous blogs, for instance in reference 10, together with a discussion to retract all these papers, en masse, be­cause they can be harmful: “Neuroscientists working with such controversial populations need to be espe­cially careful in analyzing their data, and aware of how their work may be used in a broader social con­text [10,11].”

References

1. Rinck PA. Functional Imaging. In: Rinck PA. Magnetic Resonance in Medicine. The Basic Textbook of the European Magnetic Resonance Forum. 12th edition; 2018|2020.
2. Belliveau JW, Rosen BR, Kantor HL, et al. Functional cerebral imaging by susceptibility-contrast NMR. Magn Res Med 1990; 14: 538-546.
3. Ogawa S, Lee TM, Kay AR, Tank DW. Brain magnetic reso­nance imaging with contrast dependent on blood oxygenation. Proc Natl Acad Sci USA 1990; 87: 9868-9872.
4. Kim S-G, Ogawa S. Biophysical and physiological origins of blood oxygenation level-dependent fMRI signals. Journal of Cerebral Blood Flow & Metabolism 2012; 32: 1188–1206; doi:10.1038/jcbfm.2012.23.
5. von Schulthess G. Clinical MR in the year 2010. Mag Res Med 1999; 8: 133-145.5.
6. Bennett CM, Baird AA, Miller MB, and Wolford GL. Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for proper multiple comparisons correction. Journal of Serendipitous and Unexpected Results 2010; 1(1): 1–5.
7. Bennett CM, Miller MB. fMRI reliability: influences of task and experimental design. Cogn Affect Behav Neurosci. 2013;13(4): 690-702. doi: 10.3758/s13415-013-0195-1.
8. Ponseti J, Granert O, Jansen O, Wolff S, Beier K, Neutze J, Deuschl G, Mehdorn H, Siebner H, Bosinski H. Assessment of pedophilia using hemodynamic brainresponse to sexual stimuli. Arch Gen Psychiatry 2012; 69: 187-194.
9. Ponseti J, Granert O, van Eimeren T, Jansen O, Wolff S, Beier K, Deuschl G, Bosinski H, Siebner H. Human face processing is tuned to sexual age preferences. Biology Letters 2014; 10(5), 20140200 DOI: 10.1098/rsbl.2014.0200
10. Bor D. The dilemma of weak neuroimaging papers. www.­danielbor.com/dilemma-weak-neuroimaging.
11. The Neurocritic. Let's face it: Publishing weak data on face processing in pedophiles is pointless. http://neurocrit­ic.blogspot.fr/2014_05_01_archive.html

Citation: Rinck PA. Functional charlatans. Rinckside 2015; 26,4: 9-11.

A digest version of this column was published as:
Rinck PA. The functional charlatans.
Aunt Minnie Europe. Maverinck. 7 April 2015.