Examples · network meta-analysis

From extracted arms to a ranked treatment network

This example follows one review from arm-level extraction to a saved network meta-analysis. It reproduces part of the 2018 update of a Cochrane network meta-analysis of anti-VEGF treatment for diabetic macular oedema (CD007419). The example uses three randomised trials and two efficacy outcomes at 12 months.

Use it to see what each step asks of you, what the app checks, and which decisions stay yours. The numbers are real results from this example review. They are not a substitute for the published review. For the method itself, its assumptions and its limits, read Methodology · network meta-analysis.

3 trials6 treatments2 outcomes · 12 monthsCommon effect

NOTE · What this example covers

Three trials (DRCR.net Protocol T, RESTORE and RESOLVE), six treatments, and two outcomes: the proportion of eyes gaining 15 or more ETDRS letters, and mean change in visual acuity, both at 12 months. The 2018 update pools 24 trials, so expect wider intervals here and no estimate of heterogeneity or inconsistency.

NOTE · Before you start

Network meta-analysis must be turned on in the review's settings. If it isn't, Network setup says so and the run is refused.

The path at a glance

Step	Where	What you decide
1	Extraction	The randomised arms of each trial, their results, and the source passage for every value
2	Analysis · Network setup	For each outcome: model, effect measure, direction, follow-up window and reference treatment
3	Analysis · Network setup	Which treatment group each arm belongs to, whether to merge groups, and when to refresh the evidence
4	Analysis · Network setup	Whether the outcome is ready to run
5	Analysis · Results	How to read the estimates, the ranking and the diagnostics
6	CINeMA	The certainty of each network comparison

Step 1 of 6

Extract each trial as randomised arms

A network meta-analysis compares treatments through the arms of each trial, so each trial is entered as one study with all of its randomised arms. Don't split a three-arm trial into separate two-arm comparisons. The network needs to know that the arms share one randomisation.

Open Extraction, choose a study, and switch from Verify to Form. The form for a network review has four parts:

Study arms: one entry per randomised arm, with its label, treatment, dose, route, regimen and randomized N. The randomized N is roster information. It stays separate from the number analysed for each outcome.
Outcome and population: the analysis set, the timepoint, and how missing data were handled for this outcome.
Results by arm: for each arm, the result for this outcome. Use events and outcome N for a binary outcome, or mean, SD and outcome N for a continuous one. Each arm also records an Outcome disposition (Results entered, Pooled with other arms, Not reported, or Excluded with reason) and where its denominator came from.
Source review: every value links to a quote in the paper or supplement. A single quote can back several fields. In RESOLVE, one sentence on randomisation backs 19 fields across the three arms.

RESOLVE in Verify mode showing its outcomes and review state — Fig 1.1Verify shows each outcome's review state for the study. RESOLVE has two validated outcomes.

Form with one source quote backing 19 arm fields — Fig 1.2One quote from the methods section backs the arm labels, doses, routes, regimens and randomized Ns of all three arms. The reviewer confirms each use.

Results by arm with events 18 and outcome N 51 — Fig 1.3Results by arm: 18 of 51 eyes gained 15 or more letters in the first ranibizumab arm. Events and outcome N each carry their own source.

Supplement page with numbered quote pins — Fig 1.4Quotes are pinned, numbered, to the page they came from, including supplements. On this page of Supplement 2, pin 34 marks the 18 events in RESOLVE's results table and pin 33 the outcome N of 51.

The arm data in this example

Mean change in visual acuity at 12 months (ETDRS letters):

Trial	Arm	Mean	SD	Outcome N
Protocol T	Aflibercept 2.0 mg	13.3	11.1	208
Protocol T	Bevacizumab 1.25 mg	9.7	10.1	206
Protocol T	Ranibizumab 0.3 mg	11.2	9.4	206
RESTORE	Ranibizumab 0.5 mg + sham laser	6.8	8.3	115
RESTORE	Ranibizumab 0.5 mg + laser	6.4	11.8	118
RESTORE	Sham injection + laser	0.9	11.4	110
RESOLVE	Ranibizumab 0.3–0.6 mg	11.8	6.6	51
RESOLVE	Ranibizumab 0.5–1.0 mg	8.8	11.0	51
RESOLVE	Sham	−1.4	14.2	49

TIP · Validate before you analyse

Network setup only uses validated rows. Draft rows don't block other outcomes, but they never enter a network. Validate each outcome in Extraction first. If Network setup says studies need arm adoption, those studies were extracted before arm-level entry. Adopt their saved arms in Extraction, review the quotes, and validate.

NOTE · Multiple reports of one trial

RESTORE has three reports: the 12-month paper, the registry record and the 3-year extension. Linking them as Reports of this trial makes the network count the trial once.

Learn moreMethodology · network meta-analysis assumptions

Step 2 of 6

Choose network settings for each outcome

Open Analysis and the Network setup tab. Start with Prepare treatment-group review. It proposes a treatment group for each arm and doesn't fit a model or change any measurements. You confirm those groups in Step 3.

Step 1: Check the data lists every outcome in a settings table. Each outcome has its own settings, and they apply to every analysis of that outcome:

Setting	15-letter gain	Visual acuity change
Model	Common effect	Common effect
Effect measure	Risk ratio	Mean difference
Better when	Higher	Higher
Time window	52 ± 8 weeks (44–60)	52 ± 8 weeks (44–60)
Reference	Aflibercept 2.0 mg	Aflibercept 2.0 mg

Network settings table for each outcome — Fig 2.1Each outcome has its own model, measure, direction, follow-up window and reference. The adverse-event outcomes can't run yet because they have no follow-up window.

The direction matters: the ranking is ordered by it. The time window decides which reported timepoint each trial contributes. Protocol T reports at one year and RESOLVE and RESTORE at month 12, and all three fall inside 44–60 weeks.

WARN · Why common effect here

The three trials give five independent comparisons: two each from the three-arm trials, and one from RESOLVE once its ranibizumab arms are pooled. They are used to estimate five treatment effects against the reference. No evidence is left over, so the network has zero residual degrees of freedom and between-study variance can't be estimated. If you choose random effects for such a network, Check network stops and asks you to choose a common-effect model explicitly. The results then say plainly that heterogeneity and inconsistency are not estimable. That is not evidence that the trials agree.

Step 3 of 6

Confirm treatment groups

Step 2: Confirm treatment groups decides which network node each arm counts as. The Grouping card sets the starting point:

Each distinct intervention (used here, 16 groups). Arms are grouped only when they match on treatment, dose, schedule, regimen, route and co-interventions.
By treatment (6 groups) · By treatment and dose (8) · Biosimilars with their reference product (13) · Ignoring co-interventions (13).

Each run records the grouping it was fitted under.

Below the card, Treatment groups lists every group with its arms. Work through the Needs you filter until it reads 0. Every arm the network uses must be in a confirmed group. Groups can be empty: here 16 groups exist, but only six hold arms for these outcomes, so the network has six treatments. Empty groups play no part in a run.

Grouping card and treatment-group review — Fig 3.1The grouping choice, and the confirmation that every study arm is assigned to a confirmed treatment group.

A reviewer decision: one ranibizumab node

With exact grouping, the three trials used three different ranibizumab regimens: 0.3 mg in Protocol T, 0.5 mg with sham laser in RESTORE, and two dose-escalation arms in RESOLVE. They don't share a node, so the network falls into separate parts. Like the published review, this example compares treatments at drug level, so the reviewer merged them into one group named Ranibizumab, any dose.

Merge into… shows the consequence before saving. Here the first merge takes the network from three parts to two.
RESOLVE's two ranibizumab arms then land in the same group. The app asks you to confirm that they are pooled with other arms and to give a reason. Its combined values (n = 102, mean 10.3) match the pooled column RESOLVE reports itself.
The merged group lists 8 entries from 3 studies: four ranibizumab arms, each with a result for both outcomes. The group name records the assumption.

The merged ranibizumab group with eight arms from three studies — Fig 3.2Each arm in the merged group shows its study, outcome, N and original label. Change group or Leave out works on one arm at a time.

WARN · Merging is an assumption

Merging doses treats them as one treatment. Record the reason, and consider a sensitivity run under By treatment and dose to see whether the estimates and ranking hold.

RESTORE's Ranibizumab 0.5 mg + laser arm stays a separate node because its laser co-intervention makes it a different intervention under this grouping.

Refresh the evidence, then read the data summary

Once the groups are confirmed and every outcome you want to run has its settings, choose Refresh validated evidence in Step 1. Refresh doesn't fit a model. It rebuilds what each outcome gives the network from the validated rows and confirmed groups. Refresh again after any change to groups or validated values. In this example the reviewer refreshed straight after the ranibizumab merge.

Then, under Data for the network, choose the outcome to analyse. The summary shows how many studies, validated rows and treatments it has, whether the network is connected, and whether any arm is missing numbers. Check that these counts match what you expect.

Data for the network summary — Fig 3.3Three usable studies, six treatments, connected, no arms missing numbers. Linked reports of one trial are listed and count once.

Step 4 of 6

Check the network

Under Step 3: Run the analysis, choose Check network. The check stops the run and tells you what to fix when:

the review isn't set up for network meta-analysis;
the outcome has no reference treatment, or the reference isn't in the network;
the outcome has no direction (higher or lower is better) or no follow-up window;
any result the outcome draws on isn't validated;
random effects are chosen but the network can't estimate between-study variance;
two reports of one trial both contribute results at the same follow-up.

Check network reporting that the network passed all checks — Fig 4.1“The network passed all checks.” The outcome being run is named above the buttons.

Make sure the outcome named on the card is the one you mean to run before you start the run.

Step 5 of 6

Run and read the results

Choose Run analysis. Once an outcome has a run, the button reads Run again. The visual acuity run here took about 15 seconds. When it finishes, the Results tab opens on the saved run. Runs stay Draft until you choose Lock this outcome, and earlier runs, including failed ones, stay in the run history.

If the run header says Some results were left out, open Network setup and review the treatment groups before relying on the result.

Finished run header and network graph — Fig 5.1The run header states the model, the limits, the measure, the reference and the grouping. Circle size shows participants, line width shows studies, and shaded areas are multi-arm trials.

League table

Every treatment against every other. The lower-left half holds network estimates and the upper-right half holds direct estimates. For this outcome higher is better, so a value above 0 favours the row treatment.

Selected network estimates, mean difference in letters (95% CI):

Comparison	Mean difference
Aflibercept vs ranibizumab, any dose	2.10 (0.12 to 4.08)
Aflibercept vs bevacizumab	3.60 (1.56 to 5.64)
Ranibizumab, any dose vs laser	5.90 (3.28 to 8.52)
Aflibercept vs laser	8.00 (4.72 to 11.28)
Bevacizumab vs laser	4.40 (1.18 to 7.62)
Ranibizumab, any dose vs sham	11.70 (7.35 to 16.05)

For the 15-letter outcome (risk ratio), the example's estimates were close to the 2018 update's, with wider intervals. The review reports this outcome as gaining 3 or more lines, which is 15 letters:

Comparison	This example	2018 Cochrane update
Aflibercept vs laser	3.57 (1.69 to 7.69)	3.66 (2.79 to 4.79)
Bevacizumab vs laser	2.50 (1.15 to 5.26)	2.47 (1.81 to 3.37)
Ranibizumab, any dose vs laser	2.76 (1.36 to 5.63)	2.76 (2.12 to 3.59)
Ranibizumab, any dose vs aflibercept	0.77 (0.59 to 0.99)	0.75 (0.60 to 0.94)

A risk ratio above 1 favours the first treatment, because a gain of 15 or more letters is a good outcome.

Ranking

P-score and SUCRA, ordered by the outcome's direction:

Rank	Treatment	P-score	SUCRA
1	Aflibercept 2.0 mg	0.98	98%
2	Ranibizumab, any dose	0.72	72%
3	Ranibizumab 0.5 mg + laser	0.64	64%
4	Bevacizumab 1.25 mg	0.46	46%
5	Laser photocoagulation	0.20	20%
6	Sham injection, no laser	0.00	0%

Rank probabilities, consistency and transitivity panels — Fig 5.3Rank probabilities, the consistency panel and the transitivity reminder.

WARN · A ranking is not certainty

Read the ranking alongside the effect estimates and their intervals, risk of bias, and the certainty review. In this network, Consistency reports that inconsistency diagnostics are not estimable. Transitivity needs your judgement about effect modifiers across comparisons.

Keeping and sharing the result

The network graph, league table, ranking, rank probabilities and consistency panels each have a Download button.

If evidence or settings change after a run, the outcome's status reads Results are out of date until you run it again. When an outcome's analysis is final, choose Lock this outcome. Locking ties the outcome to the method of that completed run, and the run button then reads Locked. If the evidence changes after locking, start a new analysis version to run it again.

Learn moreMethodology · reading network results Learn moreMethodology · ranking

Step 6 of 6

Rate certainty with CINeMA

Open CINeMA. Each outcome has a tab, and each network comparison has a row. This example has 15 comparisons per outcome. Rate six domains for each row:

Within-study bias · Reporting bias · Indirectness · Imprecision · Heterogeneity · Incoherence

Every rating starts as Not assessed. Next to the ratings, the app notes what it can see, for example that an interval excludes no effect, or that heterogeneity and incoherence are not estimable in this network. Once you rate the domains, it suggests an overall certainty: one level down for each “some concerns” and two for each “major concerns”. The ratings are yours. You can change the suggestion with a reason, and nothing is decided automatically. With 15 comparisons and six domains, each outcome here needs 90 domain ratings. Save as you go, and finalise when every comparison is rated.

CINeMA certainty table with network limits and domain ratings — Fig 6.1The network's limits sit at the top, and every comparison shows its effect and six domain ratings. Finalising counts the outcome towards the analysis freeze.

Learn moreMethodology · network meta-analysis certainty

What this example shows, and what it doesn't

It shows

trials entered once, with all their randomised arms, and every value traceable to a quote;
a drug-level grouping decision made explicitly and recorded on the group;
a connected network checked before fitting, and results that state their own limits.

It doesn't

reproduce the 2018 update. Three of its 24 trials are used, so intervals are wider and heterogeneity and inconsistency can't be assessed;
include trials that randomise both eyes of one participant and report adjusted analyses, such as DRCR.net Protocol I. Independent arm-level data isn't available for them in this form;
finalise certainty. The CINeMA ratings are still to be made.

Learn moreValidation & limitations · network meta-analysis

Overview

Examples

Validation & limitations

Resources