Study design

  • There are different types of scientific studies. In increasing scientific evidence are:
  1. Cell culture at petri dishes or in a test tube, also called in vitro study. In vitro is Latin for glass.
  2. Animal study or in vivo study. In vivo is Latin for life.
  3. Human study or clinical trial. These are studies in humans. There are several human studies. These are:
    1. Case-control study.
    2. Cohort study.
    3. Cross-sectional study.
    4. Crossover study.
    5. Clinical trial or intervention study (RCT). A good intervention  study is always placebo-controlled and is divided into:
      • Open intervention study.
      • Single blind intervention trial.
      • Double-blind intervention study (gives direct cause-effect relationship). This type of study has the highest burden of scientific proof.
  • A review of scientific studies. A review article combines all scientific studies on a particular nutrient, based on that a conclusion can be drawn. Because it is often the case that in one study an association was found while in other not. This has to do with the unconscious errors and the statistical methods that researchers had used.
  • The purpose of a review article is to find a clear answer about a certain topic.
  • A special form of review is meta-analysis.
    Meta-analysis is a statistical technique for combining the findings from independent studies.
    Meta-analysis of trials provides a precise estimate of treatment effect, giving due weight to the size of the different studies included.
  • A meta-analysis can be conducted with STATA software.
  • The conclusion of meta-analysis is reliable when it is free of heterogeneity and publication bias.
  • Publication bias is a type of bias that occurs in published academic research. It occurs when the outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it.
  • A funnel plot is a graph designed to check for the existence of publication bias; funnel plots are commonly used in systematic reviews and meta-analyses. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly funnel-shaped distribution.
  • The tendency towards publication bias is greater for observational studies than for randomized clinical trials.
  • The influence of a potential publication bias on findings can be explored by using the Duval and Tweedie trim-and-fill procedure.
  • Publication bias is assessed statistically by using Egger's and Begg's tests and visual inspection of funnel plots.
  • Publication bias will result in asymmetry of the funnel plot.
  • Publication bias is present when the p-value of the Egger’s test or Begg’s test is lower than 0.05.
  • To correct for publication bias, Duval and Tweedie's trim-and-fill analysis should be used.
  • Trial sequential analysis is a methodology that can be used in systematic reviews and meta-analyses to control random errors and to assess whether further trials need to be conducted.
  • Heterogeneity in meta-analysis refers to the variation in study outcomes between studies.
  • The (between-study) heterogeneity can be assessed by using Cochran's Q and the I2 statistic.
  • I2 values of 25%, 50%, and 75% were considered to represent low, moderate and high heterogeneity, respectively.
    When I2 > 50%, possible publication bias was assessed by examining the asymmetry of funnel plots or using Egger’s test.
  • Potential explanatory sources of heterogeneity can be explored by using meta-regression.
  • A fixed-effect model is used to estimate the pooled risk, like RR or OR when there is no evidence of heterogeneity; otherwise, a random-effect model will be conducted.
  • Sensitivity analysis is used to investigate the effect of a single study on the outcome measures.
  • The quality of the evidence in the studies can be assessed by The Newcastle-Ottawa Scale (NOS) score. The NOS score above 6 is regarded as a high-quality study.
  • Randomized, placebo-controlled intervention study (RCT) provides a causal correlation but not a realistic view on a diet because we do not eat nutrients but food items providing the necessary nutrients.
  • Patient-control studies and cohort studies show no causal correlation but a realistic view on a diet.
  • The results found in a meta-analysis of cohort studies are more reliable than in patient-control studies.
  • The dietary recommendations of European Food Safety Authority (EFSA) are based on the conclusions found in review articles of randomized, placebo-controlled intervention studies and review articles of cohort studies.
  • For non-food researchers it is not sensible to believe the conclusion of one simple study without knowing the advantages and disadvantages of that study.
  • Odds ratio is most commonly used in case-control studies, while relative risk is most common used in cohort studies.
  • Relative risk (RR) or odds ratio (OR) of 1 means no risk.
    Relative risk (RR) or odds ratio (OR) of 1.56 means an increased risk of 56%.
    Relative risk (RR) or odds ratio (OR) of 3.56 means a higher risk of 256%.
    Relative risk (RR) or odds ratio (OR) of 0.45 means a lower risk of 55%.
  • 95% CI of 1.34-2.41 means when the study was repeated 100 times, at least 95 times a value was found between 1.34 and 2.41.
    A 95% confidence interval (95% CI) is often interpreted as indicating a range within which can be 95% certain that the true effect lies.
    Larger studies tend to give more precise estimates of effects (and hence have narrower confidence intervals) than smaller studies. Thus, the 95% confidence interval (CI) is used to estimate the precision of the risk (RR or OR).
  • There is statistical significance when the relative risk of 1 is not found in 95% CI or the p-value of the study is less than 0.05.
    Statistically significant is not necessary also mean clinically significant.
  • P > 0.05 means not enough evidence to reject the null hypothesis.
    The null hypothesis is always, there is no association/relationship between a disease and a certain nutrient.
  • If you (a consumer) want to know whether a certain nutrient (e.g. vitamin D) is healthy for the body or not, read a review article of randomized, double-blind, placebo-controlled intervention studies.
  • If you (a consumer) want to know whether a diet is healthy for the body or not, read a review article of cohort studies.
  • Each study design has advantages and disadvantages.
  • The advantages and disadvantages of different study designs are:
    1. Randomised controlled trial (RCT) is an experimental comparison study in which participants are allocated to treatment/intervention or control/placebo groups using a random mechanism. RCT has the highest burden of scientific proof.
      • Advantages:
        • Unbiased distribution of confounders.
        • Blinding more likely.
        • Randomization facilitates statistical analysis.
      • Disadvantages:
        • Expensive: time and money.
        • Volunteer bias.
        • Ethically problematic at times.
  1. Crossover design is a controlled trial where each study participant has both therapies, e.g, is randomised to treatment A first, at the crossover point they then start treatment B. Only relevant if the outcome is reversible with time, e.g, symptoms. Crossover design can be used in cholesterol or blood pressure study.
    • Advantages:
      • All subjects serve as own controls and error variance is reduced thus reducing sample size needed.
      • All subjects receive treatment (at least some of the time).
      • Statistical tests assuming randomization can be used.
      • Blinding can be maintained.
    • Disadvantages:
      • All subjects receive placebo or alternative treatment at some point.
      • Washout period (to eliminate the carryover effect) is lengthy or unknown.
      • Cannot be used for treatments with permanent effects.
  1. Cohort study is a study design in which data are obtained from groups who have been exposed or not exposed, to the new technology or factor of interest (e.g. from databases). No allocation of exposure is made by the researcher. Cohort study has often less errors than case-control study and therefore, the conclusion of cohort study is more reliable than case-control study.
    • Advantages:
      • Ethically safe.
      • Subjects can be matched.
      • Can establish timing and directionality of events.
      • Eligibility criteria and outcome assessments can be standardized.
      • Administratively easier and cheaper than RCT.
    • Disadvantages:
      • Controls may be difficult to identify.
      • Exposure may be linked to a hidden confounder.
      • Blinding is difficult.
      • Randomisation is not present.
      • For rare disease, large sample sizes or long follow-up is necessary.
  1. Case-control study is a study design in which patients with a certain outcome or disease and an appropriate group of controls without the outcome or disease are selected (usually with careful consideration of appropriate choice of controls, matching, etc) and then information is obtained on whether the subjects have been exposed to the factor under investigation.
    • Advantages:
      • Quick and cheap.
      • Only feasible method for very rare disorders or those with long lag between exposure and outcome.
      • Fewer subjects needed than cross-sectional studies.
    • Disadvantages:
      • Reliance on recall or records to determine exposure status.
      • Confounders.
      • Selection of control groups is difficult.
      • Potential bias: recall and/or selection.
  1. Cross-sectional survey is a study that examines the relationship between diseases (or other health-related characteristics) and other variables of interest as they exist in a defined population at one particular time (e.g. exposure and outcomes are both measured at the same time). Best for quantifying the prevalence of a disease or risk factor and for quantifying the accuracy of a diagnostic test.
    • Advantages:
      • Cheap and simple.
      • Ethically safe.
    • Disadvantages:
      • Establishes association at most, not causality.
      • Recall bias susceptibility.
      • Confounders may be unequally distributed.
      • Neyman bias.
      • Group sizes may be unequal.