1.9Certainty

What it does

Once an outcome has been pooled, the Certainty stage prepares a rating of how much confidence the evidence deserves. A pairwise review gets GRADE: a row for each outcome of each completed analysis, each of the five domains derived from that run (the confirmed risk-of-bias assessments, the heterogeneity, the events and the confidence interval), with the basis of each derivation written out. A network review gets a CINeMA-style review, one row per comparison, six domains each rated no, some or major concerns. A review whose evidence is coded gets CERQual, where enabled for your workspace: four components per finding, proposed from the CASP appraisals and the coded data. In every case the AI's rating is a draft. A person opens it, changes any domain with a reason, chooses the overall certainty, and finalises it; a finalised rating shows who finalised it and when, and is never overwritten by a later run. When screening, extraction or risk of bias changes after a rating was made, the page says so and marks the rating's row Inputs changed; the rating itself is not derived again. The rows assemble into the Summary of findings table the report carries.

www.axelium.aiGRADE

1. 11. Each rating’s final certainty, after the five domains and any upgrades.
2. 22. Draft, In review or Finalised; a finalised row shows who finalised it.
3. 33. The Summary of findings leaves as a file from here.

What you do

GRADE for a pairwise outcome

A rating reads Draft, In review or Finalised. Open a row to see each domain's severity, the basis written for it and the signals it was derived from; select Start review, or change a domain, and Finalise appears. Finalise asks you to confirm, and the rating then counts towards the Analysis freeze milestone. The method and thresholds are on the GRADE methodology page.

www.axelium.aiGRADE

1. 11. Where the rating starts, and why, kept as written when it was derived.
2. 22. Each factor’s rating and its basis; changing one needs a reason and is kept as an override.
3. 33. Start review, then Finalise; both ask you to confirm.

What a rating is derived from

Every automatically derived domain carries its basis as text, and that text is part of the record: it is never rewritten once stored, so a rating derived last month still says what it rested on then.

• Risk of bias reads the review's primary appraisal checklist, weighs each study by its inverse-variance weight in the pooled analysis and counts only assessments a reviewer has confirmed. The domain is rated down once when studies at high risk carry at least 40% of the confirmed weight, and twice at 70%. The basis says how many studies it rests on; with no confirmed assessment it stays not serious and sends you to the Risk of bias review queue.
• Inconsistency reads the heterogeneity of the run and is not rated down when there are too few studies for the statistic to mean anything.
• Indirectness is never rated down automatically; only a reviewer can.
• Imprecision counts the events in the studies pooled for that outcome and checks the pooled 95% confidence interval on the scale the result is reported on, against no effect and, when one is set for the outcome, against the minimal important difference.
• Publication bias is not rated down below ten studies; at ten or more it is rated down when the asymmetry test is positive, and also when no test has been run, until a reviewer clears it after checking the funnel plot.

Each domain also records whether it was set automatically or by a reviewer, so the Summary of findings never presents a machine judgement as a person's.

Network certainty

Pairwise GRADE is not forced onto network estimates. Axelium prepares one row per network comparison for a CINeMA-style review: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence, each rated no, some or major concerns, and a concern needs a reason. Beside each rating, the evidence hints describe what the numbers say; they never set a rating, and unavailable diagnostics are never read as reassuring. A suggested overall certainty drops one level for each “some concerns” and two for each “major concerns”; you can choose another level with a reason. Finalising asks you to confirm, records who finalised and when, locks the outcome when your role may lock analyses, and counts it towards the Analysis freeze. See certainty review and the worked example.

CERQual for a qualitative finding

Where enabled for your workspace, GRADE-CERQual rates how much confidence to place in each review finding against four components: methodological limitations, coherence, adequacy of data and relevance. After an automated run each finding arrives with a proposed rating per component, the methodological-limitations component informed by the CASP appraisals, and the reviewer confirms, edits and finalises each one. The four judgements combine into high, moderate, low or very low confidence; each rating carries its explanation, machine-drafted while it is a proposal and the reviewer's own once finalised. A finding reads Draft, In review or Finalised; a finalised finding is never overwritten by a later run. The drawer shows the studies, codes and verbatim passages the finding rests on. Method: GRADE-CERQual confidence.

Who may finalise a rating

Finalising a rating is an approval. Each certainty point names the roles that may finalise it and the milestone it counts towards; the full list of approval points is in Approvals, locks and sign-off.

Reopen a rating

Reopening a finalised rating asks why, keeps the reason in the outcome's or finding's history together with who had finalised it and when, and returns the rating to Draft. Once the review is frozen at the Analysis freeze, ratings no longer change; the sign-off snapshot is the record.

The Summary of findings table

The GRADE page is the Summary of findings: a row for each outcome of each completed analysis, with the number of studies, each domain's severity, any upgrades, the final certainty and the row's status. The qualitative counterpart is the Summary of Qualitative Findings, one row per finding with its confidence, the explanation and the contributing studies (SoQF). Both can be pinned to the Evidence board and carried into the report; a pinned network certainty review follows later edits rather than freezing at the first pin. Pinned or not, a report draft is given the GRADE ratings, one line per outcome and timepoint, each marked finalised or provisional; an ecology review's draft is given none (What the writer draws on).

Settings

Three review settings change what a rating is derived from: the appraisal checklist marked primary (falling back to RoB 2 when none was chosen); the minimal important difference set for an outcome, given on the scale the result is reported on (such as 0.80 for a ratio); and the validated-only default of the analysis, since only confirmed assessments and the pooled run's own weights count. A rating already derived keeps its basis as it was until the analysis is run again. Settings are described in Review settings and history.

What is recorded

Limits

• A derived rating is a starting point. Automatic derivation reads the run and the confirmed appraisals; it does not read the protocol, the population or the outcome definition, so indirectness is always a reviewer's call.
• With no confirmed assessment the risk-of-bias domain stays not serious; confirm the assessments and run the analysis again.
• User-defined (custom) results never feed a certainty rating.
• An outcome summarised without pooling has no automatic certainty rating and no Summary of findings row; the review team's own judgement of its certainty goes in the summary's limitations.
• Ecology and evolution certainty is narrative; clinical GRADE and CINeMA are outside its scope.
• CERQual ratings are reviewer-owned; the review team finalises every rationale.