How TextScope reaches its findings

1. AI detection

Detection runs through a provider-independent interface. A provider returns a raw score for the “AI-generated” class, the model’s decision threshold, its name and version, and — only when it actually scored them — per-paragraph scores. Responses are schema-validated; malformed responses are discarded and reported as unavailable.

• Raw scores are not probabilities. A calibrated score is shown only when the operator has validated calibration for that model.
• Scores within ±0.15 of the threshold are reported as inconclusive.
• If paragraphs score strongly on both sides of the threshold, the result is mixed.
• The bundled tooling supports English only; other languages are reported as unsupported.
• If no provider is configured, the report says: “AI detection is currently unavailable because no inference engine is configured.” No substitute value is generated.

The reference self-hosted service uses an open-source RoBERTa classifier fine-tuned on GPT-2 output. It is a known, documented baseline and is expected to perform poorly on text from modern models; it is included so operators can run something real and evaluate it, not as an accuracy claim.

2. Model-family attribution

Attribution (OpenAI, Anthropic, Google, Meta, DeepSeek…) is a different task from detection and needs its own classifier, trained and evaluated on held-out prompts, domains and models. None has been evaluated for this deployment, so the feature is shown as unavailable. Exact model versions are not identified.

3. Writing signals (heuristics)

Measured properties of the text: sentence-length variation, average sentence length, lexical diversity (MATTR-50), stock-phrase density and repeated three-word sequences. They are explained individually and never combined into an “AI probability”.

4. Content quality

Transparent rules: Flesch reading ease, very long sentences, numbers or absolute claims without a visible source, vague appeals to authority, generic filler, presence of examples, structure and repeated words. TextScope does not verify facts or originality, and does not treat AI authorship as low quality.

5. URL audit

One public page is fetched after checking robots.txt, with private-network protection, per-hop redirect validation, a 10-second timeout, a 2 MB cap and a 25-second total budget. Up to 15 internal links are checked. JavaScript is not executed. Crawlable ≠ indexable ≠ indexed ≠ ranking: actual Google index status needs Search Console and is reported as not verified.

6. Evaluation

No accuracy figure is published until it has been measured. The repository contains an evaluation script that computes accuracy, precision, recall, F1, false-positive and false-negative rates, confusion matrices and breakdowns by domain, length and language on a labeled dataset you provide, with source-level splits to avoid contamination. See docs/textscope/MODEL_EVALUATION.md.