Part 4 · Methods reference
4.1The Axelium method
How a review is run in Axelium, stated once and versioned: the principles and the mechanisms that enforce them, what runs on its own at each stage and what a person decides, how evidence is handled and reported, how the method is validated and how it changes.
This page is the method. The user manual says what each stage does and records; the other Methods pages say why each statistical choice is sound. This page says how the two fit together, which rules are fixed and which a person decides. Cite it by version; the version history at the end is the record.
Scope and principles
The method covers a review from protocol to publication: every stage of the lifecycle table below, living updates and the sign-offs that close each part. Each principle names the mechanism that holds it.
- The AI proposes, a person decides. Every stage has a named approval point, and what the AI produces there is a draft until a person approves it or a fixed rule confirms it. The Sign-off status page counts, for each point, what a person approved, what was confirmed automatically and what still needs a person; the same counts go into the report’s appendix. See Approvals, locks and sign-off.
- Nothing protocol-level changes silently. A change that would alter the protocol or the included evidence waits on the Proposals page for a decision; approving asks for confirmation and turning down asks for a reason. What may be applied without a person is a fixed list in the product, never the AI’s own judgement of its authority. See What the AI does by itself.
- Every value traces to its source. An extracted value is stored with the quote, page and position it came from, each quote is checked against the document, and a value counts as validated only when a named person validated it. The lineage view opens the chain from a pooled result back to the passage.
- The record is the review. Decisions, who made them, when and why, the AI’s suggestion where a person overrode it, every lock and unlock and every version of the settings are kept and exported, so a reader can check the review without trusting the summary.
- Automation is bounded and disclosed. Where a rule confirms a decision without a person, the decision is marked as automatic, counted separately and can be reversed; the report states how many. The two rules are under Human oversight.
- The method is versioned. This page carries a version and a date, and a release that changes what a person checks or a decision rule adds an entry to the version history naming the version it moves to.
Lifecycle
One row per stage of a review, in the order a review passes them: what runs without a person, what a person checks before the stage counts as done, the decision rules the stage applies, the approval point that records the decision and what the record keeps. A stage with no approval point of its own is approved through the milestone that locks it.
| Stage | Automated steps | What a person checks | Decision rules | Approval point | What is recorded |
|---|---|---|---|---|---|
| ProtocolConfigure | Suggests the research question, eligibility criteria and outcomes from the protocol text; drafts the review settings. | The question, the criteria, the outcomes and the effect measures; the review mode (single or dual); who may do what. | Every setting is saved as a version; a change after results exist marks those results as made under earlier settings. | — | Each version of the settings, who saved it and when. |
| SearchSearch | Drafts the search strategy, builds a query per database, measures how much of a sample is on topic, runs the searches and removes duplicates. | The strategy before any search runs; the per-database queries and counts; the date range. | No search runs until the strategy is locked; locking it is the approval. Update searches of a living review run only the saved queries. | Search strategy | The strategy, each query as sent, what each database returned, who locked and unlocked the strategy and when. |
| ScreeningScreening | Screens every title and abstract against the criteria and gives a decision with a reason, a confidence and a verdict per criterion. | Every record it was unsure about; in dual review, every record, blind to the other reviewer; conflicts go to a lead. | An unsure record waits for a person. In dual review two decisions must agree or a lead rules; a reviewer may not rule on their own conflict. | Screening | Each decision, who made it and when, the reason, the AI’s suggestion where a person overrode it, and the PRISMA counts. |
| Full textFull text | Fetches each paper from open sources, checks whether the document is the study’s own paper, and queues the rest for the browser companion or an upload. | A paper that could not be fetched: upload it, or close the retrieval as not retrieved or as assessed on the abstract. | A text taken from PubMed Central is kept only when its front matter names the study; any other document is checked and a mismatch is flagged. “Not retrieved” is a person’s closure, never the absence of a file; assessing on the abstract needs a second person in a team review. | Full-text retrieval | Every retrieval attempt and its outcome, the document kept and why, each closure with who made it and when. |
| Full-text screeningFull-text screening | Reads the paper against each criterion, marks the passages that settle it and proposes include or exclude with the reason. | Every proposal the separate check did not confirm, every unsure study and every conflict; in dual review, every study. | In single review a clear decision is confirmed automatically only when a separate check agrees the marked passages answer each deciding criterion, and only while the setting is on; otherwise a person decides. | Full-text screening | Each decision, who made it and when, the reason and the passages, whether it was confirmed automatically, and the count of automatic confirmations the report states. |
| ExtractionExtraction | Extracts each outcome’s values with the quote, page and position they came from; checks every quote against the document; derives missing statistics where the rules allow. | Each value against its quote in the paper; every quote the check could not find; in dual review, a blind extraction of their own. | A value counts as validated only when a named person validated it; a quote counts as confirmed when a person found it in the paper, and as found when the check did. Validated or edited values are never overwritten by an automatic repair. | Extracted values, Quotes | Every value with its source quote, page and position; each validation, edit and rejection with who and when; each quote verdict; the history of each value. |
| AppraisalRisk of bias | Drafts a risk-of-bias judgement per domain with its support from the paper; drafts a CASP checklist per qualitative study. | Each judgement and its support; a published assessment may be imported instead; in dual review, their own blind assessment. | A draft binds nothing until a person validates or finalises it. Downstream certainty reads validated judgements only. Where two raters assessed the same studies, their agreement is shown. | Risk of bias, Critical appraisal (CASP) | Each assessment, its status, who validated or finalised it and when, imports, rulings on conflicts, and the agreement between raters. |
| SynthesisAnalysis | Pools the validated values with the review’s effect measure and model, runs heterogeneity, sensitivity and publication-bias checks, and for a network groups the treatments and proposes a method. | The model and effect measure; which studies entered each pool; for a network, the treatment groups and the method before results are reported; for an outcome that cannot be pooled, the reason it was not pooled and the summary of its studies by direction of effect. | Random-effects pooling is the default; the sensitivity and publication-bias checks use the REML estimator unless told otherwise. A network run is a draft until a reviewer locks its method for that outcome; a lock cannot be undone. An outcome with fewer than two rows a meta-analysis can use is summarised by counting the direction of effect, never by statistical significance; the summary is a draft until a person finalises it. | Summarised without pooling, Treatment groups, Network method | Each run with the settings it used and the values it read, the disclosure of what it fitted, the groups and who approved them, each method lock, and each summary without pooling with its reason and who finalised it. |
| CertaintyGRADE | Derives a GRADE rating per outcome from fixed rules over the pooled effect, the validated risk-of-bias judgements, imprecision and publication bias; derives CERQual ratings for qualitative findings and a certainty draft per network run. | Each factor of each rating; a reviewer may change a factor and must finalise the rating. | A rating is a draft until a person finalises it. Only finalised ratings count as approved; a changed factor is marked as the reviewer’s. | GRADE, CERQual, Network certainty | Each rating with its factors and their origin (rule or reviewer), who finalised it and when, and the run it was derived from. |
| ReportReport | Drafts the report from the pinned evidence: the PRISMA counts, a Methods section, the results, and the validation and sign-off record as an appendix. | Every sentence against the protocol and the evidence; the report is a draft until a person edits and approves it. | The PRISMA counts come from the record, never from the draft. Every generated document is labelled as generated via Axelium. Saved reports cannot change after the manuscript lock. | — | Each saved report with its date and author, earlier versions, and the appendix as it stood when generated. |
| Living updateProvenance | Runs the saved queries again over the dates since the last update, screens what is new, and lists it in the review inbox; keeps each update’s own results and ratings. | Each new study in the inbox before it enters the review; a reason when a decision goes against the AI’s screening. | Importing a study validates only the values nothing flags; a flagged value waits in Extraction. An incomplete update search is marked and its dates are searched again next time. | — | Each update search with its dates and counts, each inbox decision with its reason, and what each update changed. |
| ProposalsProposals | The automated review and the deep audit flag missing data, stale appraisals and unit issues; a narrow, well-evidenced data repair is applied and re-checked; everything else is proposed. | Each proposal on its own card: approve it, or turn it down with a reason. | What may be applied without a person is a fixed list, not the AI’s judgement; a protocol-level change always waits for approval; a repair that cannot be confirmed is handed to a person. | Proposals | Each proposal with its evidence, the decision, who made it and when, the reason given, and each automatic repair with the check it passed. |
| Sign-offsSign-off status | Counts, per approval point, what is approved, what was approved automatically and what still needs a person. | The five milestones in order — Protocol lock, Screening lock, Extraction lock, Analysis freeze, Manuscript lock — each requested and approved by an admin. | A lock freezes the stages below it; moving it back is recorded. A request shows what the milestone checks and holds until it is met. | Sign-offs | Each request, approval, refusal and unlock, who and when, and the full event-by-event export. |
| PublicationVersions | Builds the publication from the review as it stands when a curator publishes a version. | A curator of the organisation’s evidence library reviews the proposed version and publishes or declines it. | Only a curator publishes; a published version may be withdrawn by a person; a declined proposal keeps its reason in the activity log. | Publication | Each version, who published or withdrew it and when, and the sign-off record it carried. |
The second line of each stage names the page of the review where it is worked. The seventeen approval points and who may approve each are listed in Approvals, locks and sign-off.
Evidence handling
Provenance of values and quotes
Extraction reads the study’s own document — the full text where one was retrieved and checked, the registry record where the review links one — and writes each value with the sentence it came from, the page and the position on the page. A separate check then looks for every quote in the document and marks it found, found only as a close match, or not found; a reviewer answers whether the quote is in the paper, and a quote nobody could find keeps the value waiting. A value a person validates, edits or rejects keeps its history, and no automatic repair overwrites a value a person has validated or edited. See Extraction and traceability.
What is traced and what is not
The chain runs from a pooled result to the values it used, from each value to its quote, page and position, and from each quote to the document and the study. A sentence of the generated report is not traced: the report is a draft written from the pinned evidence, and the person who edits it is responsible for what it says. The PRISMA counts in a report are taken from the record, not from the draft. The limits of extraction are stated in Validation and limitations.
Reporting standard
Reports follow PRISMA 2020 in what they count and how they say it. A generated report carries:
- The flow counts. Records identified, duplicates removed, records screened and excluded, reports sought and not retrieved, reports assessed and excluded after full text, and studies included. Screening counts deduplicated study records; full text counts reports, so a second report of a study already screened is listed, not lost. “Not retrieved” is a person’s closure of the retrieval, never the absence of a file, and the appendix of reports not retrieved lists exactly those closures.
- The flow diagram. The PRISMA 2020 flow diagram drawn from the same counts, with databases and registers and other methods in their own columns: the report’s Figure 1 when the flow is pinned, and a page of its own where most numbers open the records they count.
- A Methods section. The eligibility criteria, how each outcome was defined, the effect measures and the statistical models, and how many decisions were confirmed automatically; a source-coverage summary when requested.
- The label. Every generated document says it was generated via Axelium, with the date.
- The validation and sign-off record. The last appendix: the five milestones and the approval points as they stood when the report was generated, in the Sign-off status page’s own words. A copy built for other people to read shows states and dates and no person’s name.
How reports are generated, edited, versioned and exported: Reports and exports.
Human oversight
Roles, single and dual review
A review has an admin, lead reviewers, reviewers, commenters and viewers; an organisation’s evidence library adds its owner, admins and curators. Who may approve at each point is fixed by role. A review runs in single or dual review mode. In single review the AI screens, extracts and appraises and a person checks; in dual review two reviewers decide independently and blind to each other, the AI may take one of the two seats, and a person always makes the final decision — automatic confirmation does not apply.
Conflicts and locks
When two reviewers disagree on a screening decision, an extracted value or a risk-of-bias assessment, the disagreement goes to a lead reviewer, who rules with a reason and a root cause; a lead who was one of the two reviewers may not rule. Five milestones lock a review in order — Protocol lock, Screening lock, Extraction lock, Analysis freeze, Manuscript lock. A lock freezes the stages below it, a request shows what the milestone checks and holds until it is met, and every unlock is recorded with who moved it.
What runs unattended, and how to switch it off
Once the search strategy is locked a review can run through screening, full-text retrieval and screening, extraction, appraisal, synthesis and certainty without a person, stopping at each approval point. Two rules confirm decisions on the way: in single review a clear full-text decision is confirmed when a separate check agrees the marked passages answer each deciding criterion, and the automated review applies a narrow, well-evidenced data repair after a second check verifies it. Both are marked as automatic and listed. The full-text rule has two settings, one for inclusions and one for exclusions, both on by default in single review; switch either off and every decision of that kind waits for a person. Certainty ratings are derived after each run unless the review turns that off. Everything protocol-level waits for approval regardless. See What the AI does by itself.
Validation
Three kinds of evidence stand behind the method, each public. First, a reproduction of a published meta-analysis of neoadjuvant immunotherapy in lung cancer, run through every stage from search to GRADE: three of the four pooled endpoints matched the published estimates within 6%, and overall survival was 16% off, a difference traced to one data point where the two analyses read different publications of the same trial. Read it with the discordant result, not without it: the reproduction case study. Second, agreement between raters: where two or more raters have assessed the same studies, the risk-of-bias page shows exact agreement and Cohen’s kappa (with a large-sample confidence interval) for each pair with a study-by-rater matrix, so a team can measure the AI’s drafts against its own reviewers on its own review; how to read it. Third, before a change to how queries are built ships, the saved searches of earlier reviews are replayed through it and the records each must keep are checked.
The limits are as public as the evidence: screening suggestions can be wrong, extracted values can misread a table, appraisal and certainty drafts need a person, and generated reports are drafts. See the validation posture and automation and human review.
Change control
A change is evaluated before it is released: screening, extraction and appraisal prompts run against fixed sets of examples with known answers, the search’s query building is replayed against saved production searches, and the lifecycle’s fixed rules are held by automated tests. Each automated run keeps a snapshot of the settings it ran with and the version of the instructions it used, so a result can always be read against the rules of its day. A release that changes what a reviewer sees is announced in What’s new.
The versioning rule
- A clarification, a rewording or an added example is a minor version.
- A change to what a person checks, or to a decision rule a stage applies, is a major version.
- Version 0.9 is the first public statement; version 1.0 is the owner’s sign-off of it — a change-log entry and nothing else.
- Every release that moves the version adds an entry to the version history below, naming the version it moves to, in the same release.
Version history
| Version | Date | Change |
|---|---|---|
| 0.11 | 2 November 2026 | A new approval point in the Analysis stage: an outcome that cannot be pooled is summarised by direction of effect, with the reason it was not pooled recorded, and a person finalises the summary. |
| 0.10 | 2 November 2026 | Reporting standard: a report now carries the PRISMA 2020 flow diagram, drawn from the same counts. |
| 0.9 | 31 October 2026 | First public statement of the method. |