AI Interpreting and Translation Product Quality Assurance — Pashto, Dari, and 22 More Afghan Languages

Not as a single yes or no. Safety is a property of a specific product, in a specific language and direction, in a specific setting — not of the category. A product may be acceptable for English-into-Dari wayfinding and unsafe for a Pashto consent discussion in the same hospital on the same day. What a health system needs is a machine translation risk assessment that states which is which, in writing, with the evidence attached. That is what this service produces.

There is no single answer, and any vendor that gives you one is quoting a benchmark rather than measuring your use. Pashto machine translation accuracy varies by direction, by dialect, by domain, and by whether the input is clean text or recognized speech. Published research consistently finds that accuracy falls hardest translating into English from non-European languages — which is the direction carrying the symptom, the consent, and the testimony. We measure your product on your content and report the figure with its confidence interval.

We score every shortlisted product on one rubric, using your real content rather than the vendor's demo, with two independent qualified evaluators scoring blind. The output is an AI interpreting procurement checklist you can put in a committee pack: comparative scorecards, a setting-by-setting decision memo, acceptance thresholds for the contract, and the raw scoring sheets so the conclusion can be checked against the evidence. AI translation vendor due diligence that stops at the vendor's own slide deck is not due diligence.

Speech recognition accuracy on accented and noisy clinical audio; translation quality under the ISO 5060 error typology; end-to-end speech translation fidelity and latency; integrity of numbers, doses, dates and negation; clinical and cultural register; dialect parity; and what the system does when it is unsure. Each layer is scored separately, because collapsing them into one number is what hides the failure a clinician would never catch.

Both, and they are different jobs. Text-only machine translation quality evaluation scores a written segment against a specification. Speech translation accuracy testing scores the full pipeline — recognition, translation, synthesis — as the user meets it, including latency, turn-taking, interruption handling, and behavior when the system cannot hear. A product can score well on text and fail in the room.

Reference scarcity, dialect fragmentation, and unreliable automatic metrics. There is no large trusted reference corpus to score against, so references are built as part of the engagement. Automatic metrics such as chrF++ correlate poorly with human judgment at low resource, so they run only as secondary signals. And a language-level score is meaningless where varieties diverge as sharply as Kandahari and Kabuli Pashto, so scoring resolution has to go one level deeper than it does in Spanish or French.

It depends on the product, the language, the direction, and the setting — which is why the question has to be answered with evidence rather than a policy. Some products perform acceptably on English-into-Dari for low-risk, scripted exchanges such as wayfinding or appointment confirmation. The same product may be unsafe for consent, discharge instructions, or a behavioral health assessment in Pashto. Our deployment decision memo states which is which for your specific product and setting.

Not on its own. 45 CFR 92.201(c)(3) requires review by a qualified human translator when machine translation is used for text that is critical to rights, benefits, or informed consent, or where accuracy is essential. The federal rule already draws the AI boundary; the practical question is whether your human review layer is actually catching what the machine gets wrong, which is what we measure.

Human scoring under the ISO 5060:2024 error typology, harmonized with MQM, is primary. Each segment is typed by error category, assigned a severity, and converted to a quality rating through penalty points against a threshold fixed in advance. Automatic metrics such as chrF++ and neural quality estimation run alongside as secondary signals — they are useful for regression detection and unreliable as a verdict, particularly in low-resource languages.

The share of scored segments containing a critical or major error while remaining fluent, grammatical, and confident in the target language. It is the error that a monolingual clinician, officer, judge, or teacher cannot detect because nothing about the output looks wrong. We report it separately from overall error rate because it is the figure that predicts harm.

Because a single number hides the failure. A product can hold a strong average on Pashto and break on Kandahari, and the speaker in front of your clinician is not an average. We report Kandahari, Kabuli, and eastern varieties separately, and we report the spread between best and worst as its own figure. The same applies to Kabuli and Herati Dari, and to Hazaragi as a variety of Dari.

Article 50 transparency obligations have been enforceable since August 2, 2026 and apply to providers and deployers regardless of whether the system is classified as high-risk — the Digital Omnibus deferred the high-risk regime, not Article 50. The Annex III high-risk deadline moved to December 2, 2027 and Annex I to August 2, 2028 under Regulation (EU) 2026/1744. Our compliance annex maps findings to the specific obligation and date.

No. No accredited certification scheme exists for AI interpreting accuracy in Afghan languages, and the harmonized standards that would underpin EU conformity assessment are not ready. We issue evidence — report, scoring sheets, benchmark set — not a badge. Any firm offering you a certification for this is selling something that does not exist.

Yes, and we declare it on the face of every report rather than leaving you to find it. The controls are structural: evaluators on a product evaluation are not drawn from the interpreting bench, the rubric is fixed in the specification before any output is seen, and you receive the raw scoring sheets so our conclusion can be checked against our own evidence. We do not build or resell an AI interpreting product, so no result moves revenue toward a system we own.

Sample size follows from the confidence interval you need on the reported figures, the number of language pairs and varieties in scope, and the error rate we observe in a pilot draw. We state the design before scoring begins. Real content produces a valid evaluation and demo scripts do not, so we work from your transcripts, documents, or recorded encounters under a data agreement — with a business associate agreement in place before any protected health information is touched.

LLM evaluation and benchmarking assesses a model's general capability in Afghan languages. This service assesses a shipped product — the interpreting app, the speech translation feature, the machine translation engine in your workflow — as the user actually meets it, including speech recognition, latency, failure behavior, and the human review layer around it.

Yes, and it is the more common engagement. Post-deployment work draws from live content under agreement, establishes the baseline you should have had at procurement, sets monitoring thresholds and escalation triggers, and defines the fallback protocol for handing an encounter to a qualified human interpreter.

All 24 Afghan languages, with the depth stated honestly. Pashto and Dari carry full dialect-level evaluation today. Uzbeki, Turkmeni, Hazaragi, Balochi, and Pashayi are evaluated on a scoped basis. For the smallest languages — Ormuri, Parachi, Tirahi, Prasun, and the Pamir group — the evaluator panel is named in the specification before the engagement begins, because a fill guarantee in a language with a few thousand speakers would not be honest.