Glossary: Experimental Design

A method involving animals or animal cells that tests how a substance or diet might cause ill-health in humans.

A comparative observational study in which the investigator selects people who have an outcome of interest (for example, developed a disease) and others who have not (controls). The investigator then collects data to determine previous exposure to possible causes of that outcome. Case–control studies are often reserved for early hypothesis testing or for investigating the causes of rare outcomes. Findings from case–control studies are generally considered to have lower reliability than those from prospective cohort studies.

An observational study in which a group or 'cohort' of people are observed over time in order to see who develops the outcome of interest. An approach that is often called a longitudinal study. Cohort studies differ from experimental studies, such as randomised controlled trials, because individuals effectively allocate themselves according to the extent of their exposure to the risk factor of interest. Prospective cohort studies involve following groups of people forward in time to assess who develops the outcome of interest. Conversely, in retrospective cohort studies, both the exposure and outcomes of interest all take place in the past relative to the starting point of the study.

An observational study in which the selected population is examined to see what proportion has the outcome of interest, or has been exposed to a risk factor of interest, or both. Cross-sectional studies are generally used to determine the population prevalence of outcomes or exposures. An approach that is often called a survey or a prevalence study. Cross-sectional studies can often provide useful estimates of disease burden for a particular population. However, they are less reliable for determining prevalence of very rare conditions or conditions with a short duration.  Often described as a “snapshot” of a population, cross-sectional studies cannot analyse behaviour over a period of time or determine the sequence of events.

Study and analysis of the patterns, of health, disease, behaviour or other conditions in defined populations.

Study in which investigators intervene by allocating one or more treatments (“interventions”) to certain participants, after which they observe outcomes of a pre-determined interest.

A study design widely used to help inform decisions about health care and policy. It is a way of using data to predict the effect of a particular factor or event on a system. This is done by creating a simplified computer simulation of the system. Models are typically used when it is either impossible, impractical, or unethical to create experimental conditions in which researchers can directly measure outcomes; or if a researcher is interested in the effect of something that has not yet happened (and may not happen). 

Direct measurement of outcomes under controlled conditions will always be more reliable than modelled estimates of outcomes. Most models cannot incorporate all the details of the complex natural environment. 

Study in which a researcher observes and records outcomes without providing an intervention or interfering with participant behaviour, e.g., cohort, cross-sectional or case control study. Observational studies are often loosely referred to as epidemiological studies. As observational studies are descriptive rather than experimental, they can only be used to estimate correlations not causations.

These are trials where participants (or clusters) are randomly allocated to receive either intervention or control. If well implemented, randomisation should ensure that intervention and control groups only differ in their exposure to treatment. RCTs are generally considered to be the most rigorous experimental study design. However, RCTs are not always possible in nutrition research. For example, some studies would be too long, expensive, unethical and/or unpractical to perform. 

A comprehensive survey of the research literature on a topic. Generally, the literature review is presented at the beginning of a research paper and explains how the researcher arrived at his or her research questions.

A type of literature review in which publications are discussed from a theoretical and contextual point of view, in the context of the topic area. Unlike systematic reviews, these studies do not list the types of databases and methodological approaches used to conduct the review nor the evaluation criteria for inclusion of retrieved articles during databases search.

Evaluation of scientific, academic, or professional work by independent others working in the same field. This process must be conducted for a publication to be accepted by a peer-reviewed journal.

A systematic review can be defined as a summary of the literature that uses explicit and systematic methods to identify, appraise and summarise the literature according to pre-determined criteria. The publication should clearly define the methods for reviewing the literature and extracting data and the statistical analysis plan. If performed well, systematic reviews are one of the highest levels of evidence. 

A review of previously published systematic reviews or meta-analyses. It often includes a repetition of the meta-analysis following a uniform approach. 

Problems in study design, which impact on the variables being studied. It is a deviation from the truth when collecting, analysing, interpreting and publishing the data, which can lead to false conclusions. Bias can be intentional or unintentional. 

Is a biological marker - a measurable indicator that marks or reflects a biological process, disease or event. Biomarkers are often measured and evaluated using blood, urine, or soft tissues.

In a single-blind experiment, the participants do not know whether they are receiving an experimental treatment or a placebo. In a double-blind experiment, neither the researchers nor the participants are aware of which subjects receive the treatment until after the study is completed.

A certain exposure may be associated with a disease or other outcome, without this association being causal. This can result from additional factors impacted the result. These factors arise due to methodology; such a factor is referred to as “confounder”.

The extent to which the results of a study can be applied to the general population of people; demonstrating this requires an assessment of relevant features of a study population and whether these are comparable to those in other populations.

A placebo is a substance or treatment which is designed to have no therapeutic value. A ’fake’ treatment which, if possible, is made to look the same and given in the same way as the real treatment. Placebos are used to remove bias that may arise from the expectation that a treatment produces an effect.

A process of assigning subjects to experimental or control groups in which the subjects have an equal chance of being assigned to either. Used to balance known, unknown, and difficult-to-control-for variables. The main purpose of randomisation is to avoid bias. It helps to ensure different groups being studied have similar characteristics when the study begins, allowing a fair comparison. 

Anything statistically shown to have a relationship with the incidence of a disease or other outcome. It does not necessarily mean it causes the outcome.

Any characteristic that may vary in study participants, such as gender, age, body weight, diet or behaviour. In an experiment, the treatment is called the independent variable; it is the factor being investigated. The variable that is influenced by the treatment is the dependent variable; it may change as a result of the effect of the independent variable.

The likelihood of an event occurring (e.g. development of a disease), based on factors such as age, diet, physical activity, in a person's lifetime.

An association (when one phenomenon is found to be accompanied by another) whose strength has been tested using statistical procedures. A correlation does not prove cause and effect.

The number of new cases of a disease during a given period of time in a defined population.

A statistical technique that enables the findings from multiple primary studies addressing the same research question (often identified during a systematic or umbrella review) to be combined. Its major purposes are to increase the numbers of observations and the statistical power, and to improve the estimates of the effect of an intervention or the strength of an association. 

The value and usefulness of a meta-analyses is not only based on the methodology used but also the quality of the included studies.

Analysis of the combined, original data of several individual studies.

The number of existing cases of a disease in a defined population at a specified time.

The likelihood of an event occurring (e.g., the development of a disease or condition) in a group of individuals with a different behaviour, physiology, or environment. Relative risks need to be considered with the absolute risk. 

A mathematical quantity that indicates the probability a study has of revealing a statistically significant effect. A research study must have enough power to detect an effect when there is one in reality. If a test has insufficient power to detect the anticipated effect, the investigator is unlikely to find one. If possible, a “power calculation” should be performed prior to a clinical study, to estimate how many participants are needed to be able to see an effect – if one exists at all. 

The probability of revealing an effect or association in a study sample equal or higher than the one observed, if there was actually no effect in the population. In other words, a result is called “statistically significant” if it is unlikely to have occurred by chance. The significance of a result is also called its p-value; the smaller the p-value, the more significant the result is said to be. It works on the basis of the hypothesis that if there is no effect, the results of a treatment are unlikely to have occurred. A p-value of less than 5 % (p < 0.05) means that the result would occur by chance less than 5 % of the time, and is generally considered evidence of a true treatment effect or a true relationship. A “statistically significant difference" means there is statistical evidence that there is a difference; it does not mean the difference is necessarily large, important or true.