All is not what it seems in the messy world of research
Don’t play it again, Sam

Rinckside 2021; 32,6: 17-18.

raised a long time in advance, an Ame­ri­can study into the re­pro­duci­bi­lity of re­laxa­tion time values in MRI was pub­lish­ed ear­lier this year. It was a col­la­bo­ra­tion bet­ween the US-Ameri­can Nation­al In­sti­tute of Stan­dards and Tech­no­logy (NIST) and the In­ter­na­tional Soci­ety of Magne­tic Re­so­nance in Me­di­cine (ISMRM) [1].

The study used two methods to measure T1 re­laxa­tion con­stants in phan­toms on dif­fe­rent MR machines [2]. The re­fe­rence stan­dard was an in­ver­sion re­co­very pulse se­quence and the se­cond se­quence was one of the (black box) ac­cele­rated data ac­qui­si­tion al­go­rithms com­monly used to­day on MRI ma­chines. Comparing measured values with known T1 values in a phantom was to help unravel various possible sources of distortion.

The outcome of the study demonstrated that MRI-based calculations of T1 are subject to significant bias and variation. The fast T1-mapping estimations revealed substantially greater deviations than the calculated T1 values of the inversion recovery measure­ments. The authors found that there was discrepancy between different vendors but without a consistent pattern, and stated in their evaluation that clinicians are unable to translate a — what they describe as — 'diagnostic threshold T1 value' determined on one MRI system to other MRI systems. In other words, they perceived that the general validity implied by the term ‘quantitative MRI’ is just fiction and they endorsed the scientific findings of the last four decades.

The paper was written in a rather clumsy and circuitous language, beating around the bush. One gets the impression that the results had to be presented but were not really appreciated, and the knowledgeable reader feels that the references were not selected according to importance but that the authors played dice to find whatever fitted or suited them.

spaceholder red600   Relaxation time measure­ments were con­sider­ed very im­por­tant dur­ing the first years of MR imag­ing. All ma­chines were pro­gramm­ed to create true T1 and T2 images (i.e., T1- and T2 mapp­ing), based on SE and IR se­quen­ces. After ab­solu­te T1 and T2 values had been used un­suc­cess­fully by re­sear­chers, com­bi­na­tions of T1 and T2, histo­gram tech­ni­ques, and so­phisti­cated three-di­men­sional dis­play tech­ni­ques of factor re­pre­sen­ta­tions were applied for what is called today ‘fin­ger­print­ing’ and ‘bio­mar­kers’.

Very early, standardized test objects and the protocols for their use to allow com­par­able mea­sure­ments of T1 and T2 precision and ac­cu­racy were intro­duced in the framework of an extensive European project [3]. The find­ings were so­ber­ing, but scien­ti­fi­cally pre­dict­able. In particular, the accuracy and precision with which the relaxation times T1 and T2 could be measured from the images were found to be rather disappointing and the results from different machines did not correspond with each other:

"These limitations present a considerable obstacle to the use of in vivo MR imaging to identify and characterize biological tissue … The major conclusion of the trial in respect of T1 and T2 measurement was that much work remains to be done before quan­ti­ta­tive MR imaging becomes a reality [4]."

This was known in the field for more than 30 years [5] but, strangely, the big multi-center studies heavily supported by the European Union and their follow-ups were unknown to the authors of the US-American survey since they are not cited in their papers. Still, the new results confirm and validate the outcome from the 1980s. The help­less and em­bar­rass­ing statement at the end of the American paper repeats the statement from 33 years ago:

“We suggest establishing rigorous quality con­trol pro­cedu­res for quan­ti­ta­tive MRI to promote confidence and sta­bi­li­ty in asso­ciat­ed mea­sure­ment tech­ni­ques and to enable trans­lation of mea­sure­ment thres­holds for dia­gnos­tic, disease pro­gres­sion, and treat­ment moni­tor­ing from the research center to the entire clini­cal com­muni­ty and back.”

Quality control turned out not to solve the prob­lem — from a scien­ti­fic point of view, MR imaging is a crude and not very exact tech­no­logy per se. Re­peat­ing and re­gur­gi­tat­ing studies instead of applying and understanding the existing results does not work out and will not work out; the authors are barking up the wrong tree [6]. In many instances we need tougher supervisors and referees stopping and cutting down faux research. This also includes research in artificial intelli­gence, bio­markers and finger­printing based on messy and not re­pro­ducible data. Data analytics may not be as useful in medicine as in ad­mini­stra­tive tasks and rough guesses at data in what is claimed to be pre­ci­sion medi­cine are unhealthy.

spaceholder red600   Recently, the main author of a paper met me with asto­nish­ment and in­com­pre­hen­sion and just gaped at me when I hinted that she should also read and cite ar­tic­les pub­lish­ed be­fore the year 2000 — in par­ti­cu­lar because those articles proved the results of her paper wrong. But wasn’t it clear that the results must be like that? The numbers were there. It fitted the agenda not only of this research group, but also the commercial interests behind it.


1. Keenan KE, Gimbutas Z, Dienstfrey A, et al. Multi-site, multi-platform comparison of MRI T1 measurement using the system phantom. PloS ONE. 2021; 16(6): 1-19 e0252966. doi.org/10.1371/ journal.pone.0252966
2. Stupic KF, Ainslie M, Boss MA et al. A standard system phantom for magnetic resonance imaging. Magn Reson Med. 2021; 86(3): 1194-1211.
3. EEC Concerted Research Project. Identification and characterization of biological tissues by NMR. Concerted Research Project of the European Economic Community. IV. Protocols and test objects for the assessment of MRI equipment. Magn Reson Imaging. 1988 6(2): 195-199.
4. Lerski RA, McRobbie DW, Straughan K, Walker PM, de Certaines JD, Bernard AM. Multi-center trial with protocols and prototype test objects for the assessment of MRI equipment. EEC Concerted Research Project Magn Reson Imaging. 1988; 6(2): 201-214.
5. Rinck PA. Relaxation Times and Basic Pulse Sequences in MR Imaging. in: Magnetic Resonance in Medicine. A Critical Introduction. 65-92. Off­print from the upcoming 13th edi­tion (2022/2023) of the text­book.
6. Rinck PA. Mapping the biological world. Rinckside 2017; 28,7: 13-15.

Citation: Rinck PA. All is not what it seems in the messy world of research. Don’t play it again, Sam.
Rinckside 2021; 32,6: 17-18.

A digest version of this column was published as:
All is not what it seems in the messy world of research.
Aunt Minnie Europe. Maverinck. 4 November 2021.

TurnPrevPage TurnNextPage

Rinckside • ISSN 2364-3889
is pub­lish­ed both in an elec­tro­nic and in a prin­ted ver­sion. It is listed by the Ger­man Na­tio­nal Lib­rary.


→ Print version (pdf).

The Author


Rinck is my last name, and a rink is an area of com­bat or con­test.

Rink­side means by the rink. In a double mean­ing “Rinck­side” means the page by Rinck. Some­times I could also imagine “Rinck­sighs”, “Rinck­sights” or “Rinck­sites” …
⇒ more


Bulletin Board