Pass rate, the number every vendor quotes and nobody defines

Solidus measures pass rate in its own verification pipeline. It is one of the metrics we track, and this page defines it, including why the figure you are quoted by any vendor, ours included, is close to meaningless without its denominator.

What it is, and why the definition is the whole argument

Nominally: the proportion of verification attempts that succeed.

But "attempts" is doing enormous work in that sentence, and vendors define it differently:

  • Does a person who abandoned mid-flow count as a failed attempt, or not an attempt at all? Because abandonments sit in the denominator under one definition and vanish under the other, that single choice changes the rate without anything about the product changing, see drop-off rate.
  • Do retries count once or several times? A flow that lets someone re-photograph a document until it works can report a high pass rate built on persistence.
  • Are cases sent to a manual review queue counted as passes, failures, or excluded?
  • Is it per-person or per-attempt?

A vendor quoting "98%" without answering those four questions has told you nothing. That is not a cynical observation: it is the single most useful thing on this page, and it applies to us exactly as much as to anyone else.

What moves it, other than definition

Document mix. A book of biometric passports passes at a different rate than one of worn national ID cards, and the difference is the documents, not the vendor.

Population. Age distribution, device quality, network conditions and lighting all move it, which is why a rate measured in one market transfers poorly to another.

Threshold placement. Any verification system trades false accepts against false rejects. A higher pass rate can simply mean a looser threshold, and a vendor optimising the number you asked about will happily give you one.

Why there is no figure on this page

We measure it. We will not publish it.

It would be a genuine measurement of something irrelevant to your decision, and publishing it would invite exactly the comparison it cannot support.

When there is production volume, the number goes here with its denominator, its document mix, and its threshold stated. That is what a pass rate needs to mean anything, and it is the standard we would ask you to hold every vendor to, starting with us.

What to ask instead

If you are evaluating verification vendors, the questions that produce comparable answers are: what counts as an attempt · how are retries counted · where do manual reviews land · on what document mix and population was this measured · and what does the corresponding drop-off look like.

A vendor who answers those five is telling you something. A vendor who leads with a percentage is not.

Keep reading

Powered by the Protocol

Solidus Verify is one product on the Solidus Network.

Explore the consensus, the validator economics, and the other products on the same identity layer.

Pass rate, the number every vendor quotes and nobody defines · Solidus — Solidus Verify