Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer

R. L. Thompson, E. V. Bandera, V. J. Burley, J. E. Cade, D. Forman, J. L. Freudenheim, D. Greenwood, D. R. Jacobs, R. V. Kalliecharan, L. H. Kushi, M. L. McCullough, L. M. Miles, D. F. Moore, J. A. Moreton, T. Rastogi, M. J. Wiseman

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Objective: Despite the increasing dependence on systematic reviews to summarise the literature and to issue public health recommendations, the formal assessment of the reliability of conclusions emerging from systematic reviews has received little attention. The main goal of the present study was to evaluate whether two independent centres, in two continents, draw similar conclusions regarding the association of food, nutrition and physical activity and endometrial cancer, when provided with the same general instructions and with similar resources. Design: The assessment of reproducibility concentrated on four main areas: (1) paper search and selection; (2) assignment of study design; (3) inclusion of 'key' papers; and (4) individual studies selected for meta-analysis and the summary risk estimate obtained. Results: In total 310 relevant papers were identified, 166 (54%) were included by both centres. Of the remaining 144 papers, 72 (50%) were retrieved in the searches of one centre and not the other (54 in centre A, 18 in centre B) and 72 were retrieved in both searches but regarded as relevant by only one of the centres (52 in centre A, 20 in centre B). Of papers included by both centres, 80% were allocated the same study design. Agreement for inclusion of cohort-type and case-control studies was about 63% compared with 50% or less for ecological and case series studies. The agreement for inclusion of 138 'key' papers was 87%. Summary risk estimates from meta-analyses were similar. Conclusions: Transparency of process and explicit detailed procedures are necessary parts of a systematic review and crucial for the reader to interpret its findings.

Original languageEnglish (US)
Pages (from-to)1006-1014
Number of pages9
JournalPublic Health Nutrition
Volume11
Issue number10
DOIs
StatePublished - Oct 1 2008

Fingerprint

Endometrial Neoplasms
Meta-Analysis
Food
Case-Control Studies
Public Health

All Science Journal Classification (ASJC) codes

  • Public Health, Environmental and Occupational Health
  • Nutrition and Dietetics
  • Medicine (miscellaneous)

Keywords

  • Diet
  • Endometrial carcinoma
  • Epidemiological studies
  • Reproducibility
  • Systematic literature review

Cite this

Thompson, R. L., Bandera, E. V., Burley, V. J., Cade, J. E., Forman, D., Freudenheim, J. L., ... Wiseman, M. J. (2008). Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer. Public Health Nutrition, 11(10), 1006-1014. https://doi.org/10.1017/S1368980007001334
Thompson, R. L. ; Bandera, E. V. ; Burley, V. J. ; Cade, J. E. ; Forman, D. ; Freudenheim, J. L. ; Greenwood, D. ; Jacobs, D. R. ; Kalliecharan, R. V. ; Kushi, L. H. ; McCullough, M. L. ; Miles, L. M. ; Moore, D. F. ; Moreton, J. A. ; Rastogi, T. ; Wiseman, M. J. / Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer. In: Public Health Nutrition. 2008 ; Vol. 11, No. 10. pp. 1006-1014.
@article{5d3f3a4444194985a60460e7ff99bc66,
title = "Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer",
abstract = "Objective: Despite the increasing dependence on systematic reviews to summarise the literature and to issue public health recommendations, the formal assessment of the reliability of conclusions emerging from systematic reviews has received little attention. The main goal of the present study was to evaluate whether two independent centres, in two continents, draw similar conclusions regarding the association of food, nutrition and physical activity and endometrial cancer, when provided with the same general instructions and with similar resources. Design: The assessment of reproducibility concentrated on four main areas: (1) paper search and selection; (2) assignment of study design; (3) inclusion of 'key' papers; and (4) individual studies selected for meta-analysis and the summary risk estimate obtained. Results: In total 310 relevant papers were identified, 166 (54{\%}) were included by both centres. Of the remaining 144 papers, 72 (50{\%}) were retrieved in the searches of one centre and not the other (54 in centre A, 18 in centre B) and 72 were retrieved in both searches but regarded as relevant by only one of the centres (52 in centre A, 20 in centre B). Of papers included by both centres, 80{\%} were allocated the same study design. Agreement for inclusion of cohort-type and case-control studies was about 63{\%} compared with 50{\%} or less for ecological and case series studies. The agreement for inclusion of 138 'key' papers was 87{\%}. Summary risk estimates from meta-analyses were similar. Conclusions: Transparency of process and explicit detailed procedures are necessary parts of a systematic review and crucial for the reader to interpret its findings.",
keywords = "Diet, Endometrial carcinoma, Epidemiological studies, Reproducibility, Systematic literature review",
author = "Thompson, {R. L.} and Bandera, {E. V.} and Burley, {V. J.} and Cade, {J. E.} and D. Forman and Freudenheim, {J. L.} and D. Greenwood and Jacobs, {D. R.} and Kalliecharan, {R. V.} and Kushi, {L. H.} and McCullough, {M. L.} and Miles, {L. M.} and Moore, {D. F.} and Moreton, {J. A.} and T. Rastogi and Wiseman, {M. J.}",
year = "2008",
month = "10",
day = "1",
doi = "https://doi.org/10.1017/S1368980007001334",
language = "English (US)",
volume = "11",
pages = "1006--1014",
journal = "Proceedings of the International Astronomical Union",
issn = "1743-9213",
publisher = "Cambridge University Press",
number = "10",

}

Thompson, RL, Bandera, EV, Burley, VJ, Cade, JE, Forman, D, Freudenheim, JL, Greenwood, D, Jacobs, DR, Kalliecharan, RV, Kushi, LH, McCullough, ML, Miles, LM, Moore, DF, Moreton, JA, Rastogi, T & Wiseman, MJ 2008, 'Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer', Public Health Nutrition, vol. 11, no. 10, pp. 1006-1014. https://doi.org/10.1017/S1368980007001334

Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer. / Thompson, R. L.; Bandera, E. V.; Burley, V. J.; Cade, J. E.; Forman, D.; Freudenheim, J. L.; Greenwood, D.; Jacobs, D. R.; Kalliecharan, R. V.; Kushi, L. H.; McCullough, M. L.; Miles, L. M.; Moore, D. F.; Moreton, J. A.; Rastogi, T.; Wiseman, M. J.

In: Public Health Nutrition, Vol. 11, No. 10, 01.10.2008, p. 1006-1014.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Reproducibility of systematic literature reviews on food, nutrition, physical activity and endometrial cancer

AU - Thompson, R. L.

AU - Bandera, E. V.

AU - Burley, V. J.

AU - Cade, J. E.

AU - Forman, D.

AU - Freudenheim, J. L.

AU - Greenwood, D.

AU - Jacobs, D. R.

AU - Kalliecharan, R. V.

AU - Kushi, L. H.

AU - McCullough, M. L.

AU - Miles, L. M.

AU - Moore, D. F.

AU - Moreton, J. A.

AU - Rastogi, T.

AU - Wiseman, M. J.

PY - 2008/10/1

Y1 - 2008/10/1

N2 - Objective: Despite the increasing dependence on systematic reviews to summarise the literature and to issue public health recommendations, the formal assessment of the reliability of conclusions emerging from systematic reviews has received little attention. The main goal of the present study was to evaluate whether two independent centres, in two continents, draw similar conclusions regarding the association of food, nutrition and physical activity and endometrial cancer, when provided with the same general instructions and with similar resources. Design: The assessment of reproducibility concentrated on four main areas: (1) paper search and selection; (2) assignment of study design; (3) inclusion of 'key' papers; and (4) individual studies selected for meta-analysis and the summary risk estimate obtained. Results: In total 310 relevant papers were identified, 166 (54%) were included by both centres. Of the remaining 144 papers, 72 (50%) were retrieved in the searches of one centre and not the other (54 in centre A, 18 in centre B) and 72 were retrieved in both searches but regarded as relevant by only one of the centres (52 in centre A, 20 in centre B). Of papers included by both centres, 80% were allocated the same study design. Agreement for inclusion of cohort-type and case-control studies was about 63% compared with 50% or less for ecological and case series studies. The agreement for inclusion of 138 'key' papers was 87%. Summary risk estimates from meta-analyses were similar. Conclusions: Transparency of process and explicit detailed procedures are necessary parts of a systematic review and crucial for the reader to interpret its findings.

AB - Objective: Despite the increasing dependence on systematic reviews to summarise the literature and to issue public health recommendations, the formal assessment of the reliability of conclusions emerging from systematic reviews has received little attention. The main goal of the present study was to evaluate whether two independent centres, in two continents, draw similar conclusions regarding the association of food, nutrition and physical activity and endometrial cancer, when provided with the same general instructions and with similar resources. Design: The assessment of reproducibility concentrated on four main areas: (1) paper search and selection; (2) assignment of study design; (3) inclusion of 'key' papers; and (4) individual studies selected for meta-analysis and the summary risk estimate obtained. Results: In total 310 relevant papers were identified, 166 (54%) were included by both centres. Of the remaining 144 papers, 72 (50%) were retrieved in the searches of one centre and not the other (54 in centre A, 18 in centre B) and 72 were retrieved in both searches but regarded as relevant by only one of the centres (52 in centre A, 20 in centre B). Of papers included by both centres, 80% were allocated the same study design. Agreement for inclusion of cohort-type and case-control studies was about 63% compared with 50% or less for ecological and case series studies. The agreement for inclusion of 138 'key' papers was 87%. Summary risk estimates from meta-analyses were similar. Conclusions: Transparency of process and explicit detailed procedures are necessary parts of a systematic review and crucial for the reader to interpret its findings.

KW - Diet

KW - Endometrial carcinoma

KW - Epidemiological studies

KW - Reproducibility

KW - Systematic literature review

UR - http://www.scopus.com/inward/record.url?scp=51049107720&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51049107720&partnerID=8YFLogxK

U2 - https://doi.org/10.1017/S1368980007001334

DO - https://doi.org/10.1017/S1368980007001334

M3 - Article

C2 - 18053295

VL - 11

SP - 1006

EP - 1014

JO - Proceedings of the International Astronomical Union

JF - Proceedings of the International Astronomical Union

SN - 1743-9213

IS - 10

ER -