How AI Detectors Work (and Where They Get It Wrong)

People tend to imagine AI detection as a lookup: somewhere there is a list of everything a language model has ever written, and the detector checks your text against it. That is not how any of this works, and the real answer is more interesting.

A detector never sees the model that wrote your text. It never sees a prompt. All it gets is the words on the page. Everything it reports is inferred from statistical properties of those words - properties that differ, on average, between prose a person composed and prose a model generated.

That "on average" is the whole story, including the parts where it goes wrong. Here is what happens between pasting your text and getting a number back.

The one idea underneath all of it

Language models are prediction machines. At every position, the model has a ranked list of plausible next words and picks from near the top of it. Do that a few hundred times in a row and you get prose that is fluent, coherent, and unusually likely at every single step.

People do not write that way. We reach for a word that is slightly wrong and keep it. We interrupt a clean sentence with a clause we thought of halfway through. We repeat ourselves, then over-correct. We write a nine-word sentence after a forty-word one because we got tired. Human text is full of choices a prediction machine would have rated as second-best.

Every technique below is a different way of measuring that gap.

Signal one: how surprising the words are

The oldest and most intuitive measurement. The detector runs your text through its own language model and asks, at each word: given everything before it, how likely was this word?

Average that across a document and you get something close to what the field calls perplexity. Low perplexity means the text kept choosing the obvious word. High perplexity means it kept surprising the model.

Our engine does this with a reference language model we built ourselves, from a 200,000-paragraph sample of our academic corpus - about 29.5 million words of real human papers, reports, and essays. It is a compact model rather than a large one: each word is scored against the word before it, blended with how common that word is in general. That is enough to tell "the results demonstrate a significant" from prose that actually goes somewhere unexpected, and it ships as a 2 MB file that scores a document in milliseconds, with no GPU anywhere in the loop.

The distribution matters more than the average, so we also measure its spread, its skew, how much of the text falls into the "highly predictable" bucket, and the longest unbroken run of predictable words in the whole document. A person can write one very smooth sentence. Writing forty in a row is harder.

Signal two: burstiness

Human writing has uneven rhythm. Sentence lengths swing around. Paragraphs speed up and slow down. Model output tends toward the middle: sentences cluster near a comfortable length and stay there.

This is what people mean by burstiness, and it is not one number. Our feature set measures the standard deviation of sentence length, the coefficient of variation, the range, the skew, the kurtosis, the average jump from one sentence to the next, and the proportion of unusually long and unusually short sentences. A document can look normal on the average and clearly artificial on the variance.

The same logic applies below the sentence. Clause length has a rhythm too, and so does punctuation - we measure the entropy of the punctuation marks a writer actually uses, because people have habits and models have defaults.

Signal three: curvature

This is the more modern idea, and the one that does the most work on carefully written AI text.

Perplexity asks whether your words were likely. Curvature asks something sharper: were your words likely relative to the alternatives the model would have considered at that exact position?

Text that a model generated does not just sit in a high-probability region. It sits at a local peak - almost every token is at or near the best available option, because that is the mechanism that produced it. Human text wanders around the slopes. Two documents can have identical average perplexity while one is perched on a ridge and the other is scattered across a hillside.

We compute this in closed form: for each context, the model already knows the mean and spread of the log-probabilities it would assign across possible continuations, so the actual token becomes a z-score against that distribution. No re-sampling, no repeated model calls. We then track the average curvature, its spread, how much of the text is unusually high, and the longest continuous high-curvature run.

Signal four: the stylistic fingerprint

Predictability is the core, but it is not sufficient on its own, and a detector that relies on it alone is easy to fool and prone to false alarms. So the rest of the feature set - about a hundred measurements in total - looks at style directly:

• Vocabulary diversity. Type-token ratio, hapax rate, Yule's K, and length-robust measures like MATTR and MTLD, which do not simply drift as the document gets longer.
• Sentence openers. How often sentences begin with a transition word, a pronoun, a coordinating conjunction, or the same word as the sentence before. Models are noticeably fonder of starting a sentence with However, Moreover, and Additionally than most people are.
• Repetition at the phrase level. Distinct bigram and trigram ratios, and the rate at which whole three-word sequences come back.
• Self-similarity. How much each sentence resembles the others around it, and how much that resemblance varies. Model output is more internally consistent than human writing, which drifts as the writer's attention does.
• Morphology. The rate of abstract suffixes - -tion, -ment, -ness, -ity, -ance - which rises in nominalised, formal-sounding prose.
• Known phrasing habits. A maintained lexicon of constructions that current models reach for far more often than people do - delve into, it is important to note, plays a crucial role in. This is the weakest signal in the set and the fastest to go stale, which is exactly why it is one input among a hundred rather than a rule.
• Spelling consistency. Mixing British and American spellings within one document is a rewriting artifact rather than a writing one.
• Citation density, because in academic writing the presence and distribution of references is itself informative.

None of these is decisive alone. Plenty of human writing is repetitive, formal, and fond of Moreover. That is precisely why they are combined rather than checked off.

Turning signals into a score

All of those measurements go into a trained classifier, which has learned from a large labelled corpus how much each feature is worth and how they interact. Ours is a stack: a linear head over the raw features, the curvature statistics, and the distance measurements all feed a gradient-boosted tree model that produces the final probability. The output passes through a sigmoid, softened by a temperature term so the model does not report false certainty.

Two parts of that stack are worth separating out, because they behave differently. The trained classifier learns what the AI systems in its training data look like. The curvature and distance layers do not - they only compare your text against a model of human writing. That matters, because it means part of the system still works on generators nobody has ever sampled, including ones that did not exist when the model was trained.

This is the step most explanations skip, and it is where the honesty lives: the threshold is a choice, not a discovery.

A classifier gives you a number between 0 and 1. Deciding that 0.65 means "likely AI" is a policy decision, and it is a trade. Set the bar lower and you catch more AI writing and wrongly accuse more people. Set it higher and the reverse. There is no setting where both errors go away.

We calibrate ours false-positive-first: we pick the threshold that holds wrongly-flagged human writing to a target rate, and accept whatever detection rate that leaves us. Being wrong about a machine costs nothing. Being wrong about a person costs a great deal.

Why length changes the answer

One consequence of all this deserves its own section, because it is counter-intuitive and it caught us out.

Every signal above is a statistical measurement, and statistics need data. A 90-word paragraph gives you five or six sentences - not enough sentence lengths to say anything trustworthy about variance, not enough tokens for the predictability distribution to settle down. Short human writing looks a lot like short AI writing, simply because neither has room to show its character.

Our detector therefore uses different thresholds for different lengths: a much higher bar for short text than for long. The effect on short human writing was large - false flags dropped from 8.75% to 2.75%, and for short text by non-native English speakers from 10.2% to 2.0%.

It cost us real detection ability, and we published the number: on short AI text, recall fell from 30.5% to 19.5%. We took that trade deliberately. Short text that sits under the bar is not called human - it comes back as mixed, with a score and highlights, which is a more accurate description of what we actually know.

Finding the AI parts of a mixed document

Most real documents are not entirely one thing. Someone drafts three paragraphs, generates a fourth, and edits all four.

A single document-level score cannot express that, so the detector scores every sentence individually and then decodes the sequence. Sentence-by-sentence thresholding produces nonsense - the score flickers between states every few sentences and you get confetti. Instead we run the per-sentence probabilities through a Viterbi decode with a penalty for switching states, which encodes something true about how documents get written: authorship comes in runs, not alternating sentences.

What comes out is a set of contiguous spans with character offsets, which is what the highlighting on the results page is drawn from.

Two things that tell you how much to trust the number

A probability with nothing around it invites over-reading. Two extra measurements travel with every scan.

Stability. The detector re-scores your text under a handful of harmless surface variations - the kind that change presentation without changing authorship - and reports how far the score moved. A result that holds still across all of them is a property of your writing. A result that swings is an artifact of one particular arrangement of characters, and should be weighted accordingly.

Familiarity. The engine also measures how far your text sits from the writing it was trained on, as a distance to the nearer of the human and AI clouds it learned. If your text is unlike anything in either, the model is extrapolating, and it says so rather than quietly guessing. This is why register matters: a detector built for academic prose is genuinely less reliable on a text message, and it should tell you that instead of pretending otherwise.

Where detectors get it wrong

Every honest account of this technology needs this section.

Non-native English writers are the biggest fairness problem in the field. Writing in a second language tends to produce simpler vocabulary, more regular sentence structure, and heavier reliance on learned connectives - which is, statistically, a description of the AI signal. This is not a bug one release fixes; it is inherent to measuring predictability. We track false positives on non-native writing as a standing metric across every model version specifically because it can drift upward while the headline numbers improve.

Formulaic writing looks artificial because it is. Methods sections, lab reports, legal boilerplate, and structured summaries are supposed to be uniform and predictable. That is the genre working correctly, and it reads as machine-like to a detector.

Some ordinary human prose simply scores high. When we measured this on a fresh sample of our own corpus, 7.8% of perfectly normal human academic paragraphs scored above 0.5 under the model version we had at the time. We now deliberately mine the most AI-looking human paragraphs we can find and train against them, which brought that down substantially - but it will never be zero.

Editing degrades detection. Text that was generated and then substantially rewritten sits between the two classes, because it genuinely is between them. Detectors are least certain exactly where documents are most mixed.

And the technology has an expiry date on every claim. These systems are trained on the output of the models that exist now. Newer models write differently. Any accuracy figure is a measurement against a particular set of generators at a particular time, ours included.

So what is the number for?

A detector score is evidence, not proof. It belongs in the same category as a fingerprint that partially matches or an unusual entry in a log: worth looking into, never sufficient on its own.

If you are checking your own writing, the useful reading is not "am I under the line" but where the highlights fall and why. Long stretches of uniform, predictable, transition-heavy prose read as artificial partly because they are hard to read. That is worth fixing whether or not a tool ever scans it.

If you are on the receiving end of a score about someone else's work: a percentage is not a finding. Look at the stability and familiarity readouts, look at whether the flagged spans are the formulaic sections, consider whether the author writes in English as a second language, and treat the result as one input to a conversation rather than its conclusion.

No AI detector is 100% accurate - not ours, not anyone's. Any tool that tells you otherwise is selling you something. Ours prints that warning on every page of every report it generates, precisely because those reports get forwarded to people who never saw the interface.

You can try it on your own writing at our AI Detector. It is free - a free account, no card - and it will show you its working.