Key Points:
-
Synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. This will include synthesis of study characteristics and, potentially, statistical synthesis of study findings.
-
A general framework for synthesis can be used to guide the process of planning the comparisons, preparing for synthesis, undertaking the synthesis, and interpreting and describing the results.
-
Tabulation of study characteristics aids the examination and comparison of PICO elements across studies, facilitates synthesis of these characteristics and grouping of studies for statistical synthesis.
-
Tabulation of extracted data from studies allows assessment of the number of studies contributing to a particular meta-analysis, and helps determine what other statistical synthesis methods might be used if meta-analysis is not possible.
9.1 Introduction
9.2 A general framework for synthesis
9.3 Preliminary steps of a synthesis
9.3.1 Summarize the characteristics of each study (step 2.1)
9.3.2 Determine which studies are similar enough to be grouped within each comparison (step 2.2)
9.3.3 Determine what data are available for synthesis (step 2.3)
9.3.4 Determine if modification to the planned comparisons or outcomes is necessary, or new comparisons are needed (step 2.4)
9.3.5 Synthesize the characteristics of the studies contributing to each comparison (2.5)
9.4 Checking data before synthesis
9.5 Types of synthesis
9.6 Chapter information
9.7 References
Key Points:
-
Meta-analysis is the statistical combination of results from two or more separate studies.
-
Potential advantages of meta-analyses include an improvement in precision, the ability to answer questions not posed by individual studies, and the opportunity to settle controversies arising from conflicting claims. However, they also have the potential to mislead seriously, particularly if specific study designs, within-study biases, variation across studies, and reporting biases are not carefully considered.
-
It is important to be familiar with the type of data (e.g. dichotomous, continuous) that result from measurement of an outcome in an individual study, and to choose suitable effect measures for comparing intervention groups.
-
Most meta-analysis methods are variations on a weighted average of the effect estimates from the different studies.
-
Studies with no events contribute no information about the risk ratio or odds ratio. For rare events, the Peto method has been observed to be less biased and more powerful than other methods.
-
Variation across studies (heterogeneity) must be considered, although most Cochrane Reviews do not have enough studies to allow for the reliable investigation of its causes. Random-effects meta-analyses allow for heterogeneity by assuming that underlying effects follow a normal distribution, but they must be interpreted carefully. Prediction intervals from random-effects meta-analyses are a useful device for presenting the extent of between-study variation.
-
Many judgements are required in the process of preparing a meta-analysis. Sensitivity analyses should be used to examine whether overall findings are robust to potentially influential decisions.
10.1 Do not start here!
10.2 Introduction to a meta-analysis
10.2.1 Principals of meta-analysis
10.3 A generic inverse-variance approach to meta-analysis
10.3.1 Fixed-effect method for meta-analysis
10.3.2 Random-effects methods for meta-analysis
10.3.3 Performing inverse-variance meta-analyses
10.4 Meta-analysis of dichotomous outcomes
10.4.1 Mantel-Haenszel methods
10.4.2 Peto odds ratio method
10.4.3 Which effect measure for dichotomous outcomes?
10.4.4 Meta-analysis of rare events
10.5 Meta-analysis of continuous outcomes
10.5.1 Which effect measure for continuous outcomes?
10.5.2 Meta-analysis of change scores
10.5.3 Meta-analysis of skewed data
10.6 Combining dichotomous and continuous outcomes
10.7 Meta-analysis of ordinal outcomes and measurement scales
10.8 Meta-analysis of counts and rates
10.9 Meta-analysis of time-to-event outcomes
10.10 Heterogeneity
10.10.1 What is heterogeneity?
10.10.2 Identifying and measuring heterogeneity
10.10.3 Strategies for addressing heterogeneity
10.10.4 Incorporating heterogeneity into random-effects models
10.11 Investigating
10.11.1 Interaction and effect modification
10.11.2 What are subgroup analyses
10.11.3 Undertaking subgroup analyses
10.11.4 Meta-regression
10.11.5 Selection of study characteristics for subgroup analyses and meta-regression
10.11.6 Interpretation of subgroup analyses and meta-regressions
10.11.7 Investigating the effect of underlying risk
10.11.8 Dose-response analyses
10.12 Missing data
10.12.1 Types of missing data
10.12.2 General principles for dealing with missing data
10.12.3 Dealing with missing outcome data from individual participants
10.13 Bayesian approaches to meta-analysis
10.14 Sensitivity analysis
10.15 Chapter information
10.16 References
Chapter 11: Undertaking network meta-analyses
Editors: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA
Key Points:
-
Network meta-analysis is a technique for comparing three or more interventions simultaneously in a single analysis by combining both direct and indirect evidence across a network of studies.
-
Network meta-analysis produces estimates of the relative effects between any pair of interventions in the network, and usually yields more precise estimates than a single direct or indirect estimate. It also allows estimation of the ranking and hierarchy of interventions.
-
A valid network meta-analysis relies on the assumption that the different sets of studies included in the analysis are similar, on average, in all important factors that may affect the relative effects.
-
Incoherence (also called inconsistency) occurs when different sources of information (e.g. direct and indirect) about a particular intervention comparison disagree.
-
Grading confidence in evidence from a network meta-analysis begins by evaluating confidence in each direct comparison. Domain-specific assessments are combined to determine the overall confidence in the evidence.
11.1 What is network meta-analysis?
11.1.1 Network diagrams
11.1.2 Advantages of network meta-analysis
11.1.3 Outline of this chapter
11.2 Important concepts
11.2.1 Indirect comparisons
11.2.2 Transitivity
11.2.3 Indirect comparisons and the validity of network meta-analysis
11.3 Planning a Cochrane Review to compare multiple interventions
11.3.1 Expertise required in the review team
11.3.2 The importance of a well-defined research question
11.3.3 Selecting outcomes to examine
11.3.4 Study designs to include
11.4 Synthesis of results
11.4.1 What does a network meta-analysis estimate?
11.4.2 Synthesizing direct and indirect evidence using meta-regression
11.4.3 Performing network meta-analysis
11.4.4 Disagreement between evidence sources (incoherence)
11.5 Evaluating confidence in the results of a network meta-analysis
11.5.1 Available approaches for evaluating confidence in the evidence
11.6 Presenting network meta-analyses
11.6.1 Presenting the evidence base of a network meta-analysis
11.6.2 Tabular presentation of the network structure
11.6.3 Presenting the flow of evidence in a network
11.6.4 Presentation of results
11.7 Concluding remarks
11.8 Chapter information
11.9 References
Chapter 12: Synthesizing and presenting findings using other methods
Editors: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA
Key Points:
-
Meta-analysis of effect estimates has many advantages, but other synthesis methods may need to be considered in the circumstance where there is incompletely reported data in the primary studies.
-
Alternative synthesis methods differ in the completeness of the data they require, the hypotheses they address, and the conclusions and recommendations that can be drawn from their findings.
-
These methods provide more limited information for healthcare decision making than meta-analysis, but may be superior to a narrative description where some results are privileged above others without appropriate justification.
-
Tabulation and visual display of the results should always be presented alongside any synthesis, and are especially important for transparent reporting in reviews without meta-analysis.
-
Alternative synthesis and visual display methods should be planned and specified in the protocol. When writing the review, details of the synthesis methods should be described.
-
Synthesis methods that involve vote counting based on statistical significance have serious limitations and are unacceptable.
12.1 Why a meta-analysis of effect estimates may not be possible
12.2 Statistical synthesis when meta-analysis of effect estimates is not possible
12.2.1 Acceptable synthesis methods
12.2.2 Unacceptable synthesis methods
12.3 Visual display and presentation of the data
12.3.1 Structured tabulation of results across studies
12.3.2 Forest plots
12.3.3 Box-and-whisker plots and bubble plots
12.3.4 Albatross plot
12.3.5 Harvest and effect direction plots
12.4 Worked example
12.4.1 Scenario 1: structured reporting of effects
12.4.2 Overview of scenarios 2-4: synthesis approaches
12.5 Chapter information
12.6 References
