How Pre-Check Scores Manuscripts: Why Three Odds Beat One Grade

A single letter grade tells you that your manuscript rated "B+." It tells you nothing about whether that manuscript would be accepted at a Nature sub-brand, a solid mid-tier journal in your field, or a broad open-access outlet. Science AI Journal's Pre-Check returns three numbers instead: an acceptance probability for each journal tier, along with the discipline it detected, the similar papers it found, and a confidence signal when the data is thin.

This post explains how the pipeline works, where the numbers come from, and why "60% chance at mid-tier journals" is more useful than a letter grade.

Why this matters

Researchers waste months targeting the wrong venue. A common pattern: a paper is desk-rejected from a top journal, the authors spend weeks revising for that rejection, they submit again, and desk-rejection repeats. Meanwhile, the paper would have been accepted — and read — at a highly ranked mid-tier outlet months earlier.

An acceptance estimate, grounded in what has actually been published in a 7.9M-paper local library, compresses that feedback cycle. It does not replace a full AI review or editorial judgment. It tells you what the library of published science looks like around your work — which is part of what editors use too.

What happens when you submit a title and abstract

Pre-Check runs entirely against the local library. No LLM call, no cloud model, no API round-trip for the core estimate. The pipeline has three steps.

Step 1 — Retrieval. retrieveSimilarPapers() in lib/journal-recommender/fts-retrieval.ts runs an FTS5 query against the library using BM25 ranking (Robertson & Zaragoza, 2009, doi:10.1561/1500000019) and returns up to 200 papers ranked by text similarity to your title and abstract. These are the papers most like yours that have already been published.

Step 2 — Field detection. The 200 hit IDs are joined against paper_category, which stores the LLM-classified broad field assigned to each paper at harvest time. The most frequent field across your 200 hits becomes your manuscript's detected discipline — "Engineering", "Medicine", "Computer Science", and so on. If fewer than half the hits agree on a single field, the confidence flag goes low.

Step 3 — Tier distribution. Each hit's journal is looked up in the baked metadata index. Journals are bucketed into three tiers: top (elite, selective venues), mid-tier (highly ranked, broader scope), and open/emerging (broad open-access and emerging venues). The fraction of your 200 hits in each tier — the tier density — scales an acceptance probability using per-discipline calibration anchors. The result is three percentages.

The whole pipeline runs in roughly 30–60 ms p99 on a warm cache.

Three tiers, not one verdict

The three-tier output is deliberate. A single score collapses distinctions that matter to a researcher's decision. Consider two papers:

• Paper A: top-tier probability 42%, mid-tier 71%, open/emerging 88%
• Paper B: top-tier probability 8%, mid-tier 55%, open/emerging 82%

A letter grade might put both at "B." The three-tier output tells a different story. Paper A is a genuine candidate for elite venues — a 42% acceptance chance at top journals, by the library's evidence, is not a long shot. Paper B should aim at mid-tier first; its top-tier probability sits at the base floor, and the retrieved papers offer no lift there.

The three tiers match the labels the scorer UI renders: top, mid, and open/emerging. Each has a base acceptance rate anchored to real-world averages: top journals have an approximate 8% acceptance floor in medicine; mid-tier journals sit near 40%; open-access outlets hover around 70%. The floor matters — it is what the scorer returns when the retrieved papers offer no signal to push the estimate upward.

Before May 2026, the scorer made a single Claude Haiku call per request to generate a prose verdict. The local pipeline replaced it: the probabilistic output is more precise, fully deterministic, and costs nothing to run.

Per-discipline calibration: 65% in medicine is not 65% in physics

A 65% acceptance probability means something different depending on the field. Medicine and clinical research have acceptance-rate floors near 8% at top journals and 40% at mid-tier. Physics has tighter tier separation. Social science and economics papers cluster heavily in open/emerging venues because that tier dominates those fields' publishing landscape.

Pre-Check accounts for this through discipline-calibration.json (lib/reviewers/discipline-calibration.json), a hand-curated table of per-discipline floor and gain anchors. Each entry is sourced from OpenAlex work aggregations (2020–2025), Scimago band medians, and the library's own tier counts per field:

"medicine": {
  "tier1Floor": 8,
  "tier1Gain": 70,
  "tier2Floor": 40,
  "tier2Gain": 50,
  "tier3Floor": 70,
  "tier3Gain": 25
}

The gain modulates how strongly a high tier density boosts the probability above the floor. Dense clustering of top-tier hits can push the estimate up to tier1Floor + tier1Gain — 78% in medicine. The estimate never exceeds 95%.

Compare that to computer science, where tier1Floor is 6 and tier2Gain is 48 — a tighter mid-tier band than medicine, reflecting CS's more stratified acceptance distribution. The score-calibration post covers how this per-discipline approach maps to real acceptance rates across fields.

Confidence signals and the quartile fallback

Not every paper lands in a well-covered region of the library. Two situations trigger a confidence adjustment.

Low-confidence (fewer than 10 retrieved hits). If the query returns fewer than 10 papers, or too few distinct keyword tokens can be extracted from the title and abstract, the tier distribution cannot be trusted. The probabilities collapse to the field's base rates, and the response includes lowConfidence: true. This is not a failure — it is an honest signal to the researcher that the estimate is thinner than usual.

Sparse-classified (few curated-tier hits). The curated tier list covers approximately 212 elite venues — journals where most of the library's top science was published. For papers in fields like social science, business, education, or law, those journals may not appear among the 200 retrieved hits at all. When fewer than 10 of the similar papers sit in curated-tier journals, the scorer falls back to Scimago quartiles via resolveVenueQuartile(). Q1 and Q2 journals populate the mid-tier estimate; Q3 and Q4 journals support the open/emerging estimate. Tier 1 stays at its base floor — a Scimago quartile alone does not imply the selectivity of Nature or Cell. The quartile fallback path appears in the response as scoreBasis: "quartile_fallback" in the diagnostic log.

This fallback prevents the well-known problem of hallucinated precision: rather than reporting a confident 72% built on two similar papers, the system either collapses to base rates or falls back to the broader Scimago signal.

The full tier: four LLM-backed sub-checks

The basic tier is free of LLM calls and suited for a quick library-grounded check. The optional tier=full or tier=deep mode in lib/reviewers/multi-criteria-precheck.ts runs four parallel sub-checks, each with an independent cost:

• Novelty — near-duplicate detection against the retrieved library hits
• Structure — IMRaD-component regex check (zero LLM cost)
• Methodology — discipline-specific toolkit match for ambiguous papers
• Style — gibberish and LLM-generated-text detection (zero LLM cost)

All four fire in parallel with Promise.allSettled, so a timeout in one check does not block the rest. The aggregate result is a pass when at least three of the four criteria pass. The methodology agent deep-dive explains how the same discipline-specific criteria appear in the full AI Review.

What this means for authors

Use Pre-Check early. Run it on the title and abstract you have today, before revision. If the mid-tier probability is below 40%, the library is telling you that work in your area of science has not often been published at that tier — look at what your 200 similar papers have in common, and consider whether the framing, scope, or methodology needs adjusting before targeting that venue.

Pre-Check is a venue signal, not a verdict. It answers one question: given the published record around this topic, where does this manuscript fit? The answer to whether the manuscript is correct, complete, and reproducible comes from a full review.

Run a Pre-Check on your manuscript → — it reads title and abstract only, and results come back in seconds.

If you want to explore the research landscape around your topic first, the Research Gap Finder searches free and shows what has been mined from 1,900+ citable analysis pages in the local library. Search a topic, read the gap table, and unlock the full analysis to see the research proposal it generates.

AI Review reports are written by AI agents. They are pre-submission feedback, not an editorial decision.

Frequently asked questions

Does the basic tier use a language model?

No: the default scoring path is deterministic. It runs an FTS5/BM25 retrieval against the local 7.9M-paper library, joins results against a journal-tier index, and applies per-discipline calibration anchors from discipline-calibration.json. The optional full and deep tiers, however, do run LLM-backed sub-checks for novelty, methodology, and style — so Pre-Check as a whole is not exclusively model-free.

What do the three percentages mean?

Each percentage is an estimated acceptance probability for a tier of journal: top-tier (elite, selective), mid-tier (highly ranked, broader scope), and open/emerging (broad open-access). They are grounded in where papers most similar to yours have been published, scaled by real-world acceptance-rate anchors for your detected field. They are not guarantees — the library cannot see what a specific editor will decide — but they are based on the same distribution that editors implicitly use.

Why does the detected field change the numbers?

Acceptance rates differ substantially across fields. A 50% mid-tier probability in physics and a 50% mid-tier probability in medicine are anchored to different real-world base rates: tier2Floor is 32 in physics vs. 40 in medicine. The per-discipline calibration ensures an engineering paper is not scored against medicine's acceptance distribution.

What happens when my field is not covered by the curated tier list?

For fields where most published work appears outside the approximately 212 curated elite venues — social science, business, education, law — the scorer uses Scimago quartile data as a fallback. Q1 and Q2 journals populate the mid-tier estimate; Q3 and Q4 journals support the open/emerging estimate. The response marks this path with scoreBasis: "quartile_fallback" in the diagnostics.

Is the same corpus used for Pre-Check and for AI Review calibration?

No. Pre-Check queries the local 7.9M-paper library to find papers similar to yours and uses their journal-tier distribution. AI Review's agents are calibrated on a separate corpus of 5,000+ real peer-review texts from 15 open-review platforms, including OpenReview, SciPost, PREreview, and eLife. Seven of the nine AI Review agents are calibrated on that peer-review corpus; Pre-Check reads none of it.