- Home
- Capabilities
- AI Assurance for Government
Independent AI Testing and Assurance for Government Systems in Pashto, Dari and 22 More Afghan Languages
Ariana Nexus independently tests the AI systems government agencies use in Pashto, Dari and other Afghan languages, from speech recognition and machine translation to language-model assistants, OCR and screening tools, before and after deployment. Native-written test sets and severity-weighted human scoring produce evidence your AI governance board can act on, never a pass/fail seal.
Afghan-language AI assurance at a glance
Independent testing of government AI in Pashto, Dari and 22 more Afghan languages.
- Systems tested
- Speech, translation, interpreting, assistant, OCR, search and screening AI
- Languages
- Pashto and Dari in depth; 22 more Afghan languages by tier
- Deliverables
- Test report, evidence package, model-card annex, monitoring plan
- When
- Pre-deployment AI testing, acceptance before award, and monitoring in production
- How agencies buy
- Registered in SAM.gov; direct where rules allow, or through primes and BPA holders
What is Afghan-language AI assurance?
Afghan-language AI assurance is independent testing of an AI system's accuracy, safety, security and fairness in Pashto, Dari and other Afghan languages, using native-written test sets, human baselines and severity-weighted scoring, to produce evidence an agency can use to deploy, fix, restrict, monitor or stop the system.
It is independent verification and validation (IV&V) for AI, aligned to the NIST AI RMF and its TEVV practice: an AI audit for government systems that ends in evidence, not a seal.
Under Executive Order 14434 (September 2026), new federal documents say “Super Intelligence (SI)” for AI. The testing is the same.
Federal Register, October 2026
What OMB M-25-21 requires for high-impact AI, and the evidence we produce
OMB M-25-21 compliance starts with a presumption: AI translation is high-impact when its output is legally binding or directly informs an agency decision. High-impact AI must meet seven minimum practices, required since April 3, 2026, or be safely discontinued. High-impact AI testing in Pashto and Dari supplies the evidence.
Conduct pre-deployment testing
Complete an AI impact assessment
Conduct ongoing monitoring for performance and potential adverse impacts
Ensure adequate human training and assessment
Provide additional human oversight, intervention and accountability
Offer consistent remedies or appeals
Consult and incorporate feedback from end users and the public
Practice names follow M-25-21 §4(b). Its independent review stays inside the agency; we supply evidence, not that review.
DHS’s 2025 AI use case inventory lists four translation systems as high-impact AI. Office of Management and Budget
Acquisition, model cards and language access
M-25-22 AI acquisition: vendor access for independent evaluation
M-26-04: model cards for LLMs
Justice Department language-access memo, July 2025
Documented failures of AI in Pashto and Dari
Pashto AI and Dari AI fail in documented ways. Each record names the error type our tests target.
2020
United StatesA machine translation tool changed “I” to “we” in a Pashto-speaking asylum seeker’s statement; a U.S. court denied her claim over the mismatch.Error typePronouns and grammatical personRest of World, April 2023April 2023
GermanyThe federal migration office stopped applying dialect-recognition software to Pashto. It still applies it to Dari and Persian.Error typeDialect-identification claimsVerfassungsblog, June 20262023
ResearchEnglish to Pashto, GPT-4 scored 26.7 chrF and NLLB 39.4 on the FLORES-200 benchmark.Error typePashto machine translation accuracyRobinson et al., WMT 20232025
ResearchLeading commercial models read synthetic printed Pashto at 33% to 70% word accuracy.Error typeOCR of Pashto documentsHaq, Zhang and Khan, May 20252026
ResearchSmall models often “over-corrected text to the Tehrani standard” when generating Dari.Error typeAfghan Dari, not Iranian PersianSilkRoadNLP workshop, 20262026
ResearchNo Whisper model produced Pashto-specific script in over 0.8% of test utterances. Fine-tuned models reporting about 14% word error rate scored 32.5% to 59% on new audio; the author maintains them.Error typeScript fidelity and vendor claimsRahman, arXiv, April 2026April–July 2026
United KingdomIn first-stage Home Office spot checks, 45 of 203 AI asylum case summaries (22%) failed for factual errors.Error typeSummarizer accuracyResultsense, October 2026
Pashto machine translation accuracy, English to Pashto
chrF score, higher is betterNLLB
GPT-4
chrF measures character overlap with a reference, not meaning or severity.
Robinson et al., WMT 2023Coverage gaps, as of November 2026
- Neither Azure AI Speech nor Google Cloud Speech-to-Text lists a Dari speech locale.Tests for: Persian fallback in Dari speechMicrosoft Learn, November 2026
- None of eleven multilingual safety and jailbreak benchmarks we reviewed includes Pashto or Dari.Tests for: Red-team coverageAriana Nexus research, November 2026
AI testing and evaluation (AI T&E) services for government
Every AI system an agency uses in Pashto and Dari, tested before deployment, at acquisition and in production.
Pre-deployment testing for high-impact AI
- Speech, translation, interpreting apps, assistants, summarizers, OCR, search and classification
- Native-written test sets built around your mission
- Black-box testing by querying the service, as M-25-21 allows without model access
- A human baseline from qualified interpreters and translators
- Severity-weighted analysis of which errors change a decision
Security and safety testing: AI red teaming in Pashto and Dari
- Jailbreaks, prompt injection, unsafe outputs and data leakage
- Misuse scenarios specific to your mission
Fairness and disparity testing (AI bias testing)
- Gaps against English, between Pashto and Dari, and across dialects, speakers, ages, genders and channels
- Multilingual AI evaluation where M-26-04 says cross-language benchmarks are necessary
Procurement support and model card review
- Requirements and evaluation criteria for solicitations
- Test-and-monitor clauses under M-25-22
- Vendor-claim verification and model-card review under M-26-04
- Acceptance testing before award or an option
Governance evidence for AI impact assessments
- Inputs to AI impact assessments
- Evidence packages for AI governance boards and Chief AI Officers
- Records of human oversight, remedies, appeals and feedback
Ongoing monitoring
- Re-tests and drift checks after model or vendor updates
- Sampled human review and incident review in production
Human oversight design and staff training
- When a qualified interpreter must review AI output
- How staff spot AI errors and escalate cases
National-security systems, through prime contractors
- Support aligned to the NSPM-11 test methods once published
- Personnel holding the clearances each contract requires
Published method and benchmark
- Our method, published and dated by edition
- A native-written Pashto and Dari government-AI benchmark, with a university co-author
Courts, allied agencies and state AI laws
- Machine-generated evidence for courts, should proposed Federal Rule of Evidence 707 be adopted
- Allied protection and asylum agencies
- State AI laws as they take effect

What we will never do
Independence is what an agency buys, so the limits are written down.
Never what we touched
We never test, assure or report on a system we built, deployed, sold, trained or supplied data for. The Ethics & Safety Board checks conflicts, declared on every report.No seals
No certificates, seals, badges or passes: evidence and findings only.No political work
No political or campaign work, and no testing for parties or partisan actors.No surveillance
No surveillance or targeting of any community; we build no monitoring capability.No judgments about people
No origin, identity or credibility determinations about individuals.No tainted data
No data from channels controlled by the de facto authorities in Afghanistan. Test data is native-written, licensed or consented.No overclaiming
Classified work only through prime contractors, with cleared personnel. We hold no facility clearance and claim none.Interest declared
The firm also provides human interpreting. Every report says so; no finding recommends our services.
How we test Pashto and Dari AI: method, metrics and limits
Six stages aligned to the NIST AI RMF, each ending in a deliverable. Low-resource language AI evaluation needs human judgment, so metrics never stand alone.
- 01
Scope
System, mission, users and risk. Is it high-impact?Output: Test plan - 02
Build
Linguists write mission items; the Cultural Compliance Bureau reviews them. Items stay held out.Output: Test-set specification - 03
Run
Automated metrics and blind human scoring, beside a human baseline.Output: Scored results - 04
Analyze
Severity, disparity and security findings, with confidence intervals.Output: Findings register - 05
Report
Findings, evidence package and model-card annex.Output: Test report - 06
Monitor
Re-tests, drift checks and sampled human review.Output: Monitoring plan
Inside a native-written test set
Every item is written in Pashto or Dari, never translated from English, and targets an error that changes meaning.
- Specimen 01Pronouns and grammatical personPashtoما ولیدلI saw themDariما دیدیمWe sawما is “I” in this Pashto sentence and “we” in Dari.
- Specimen 02Kinship termsPashtoد تره زویDariپسر کاکاBoth: paternal uncle’s sonEnglish says “cousin” and loses the side of the family.
- Specimen 03Names and transliterationPashtoګل احمدDariگل احمدBoth: Gul AhmadPashto writes ګ and Dari writes گ: one name to match.
- Specimen 04Military and police ranksPashtoډګروالDariدگروالBoth: colonelIranian Persian says سرهنگ; Afghan Dari does not.
- Specimen 05Solar Hijri datesPashto۱۳ لړم ۱۴۰۵Dari۱۳ عقرب ۱۴۰۵Both: 4 November 2026In Iran the same day is ۱۳ آبان.
- Specimen 06Place namesPashtoکندهارDariقندهارBoth: KandaharTwo spellings, one city.
- Specimen 07Code-switchingPashtoزما اپوینټمنټ کینسل شوDariاپوینتمنت من کنسل شدBoth: my appointment was cancelledEnglish loanwords mid-sentence must survive.
What each metric sees, and what it misses
Word error rate (WER)
chrF
COMET
Severity-weighted human scoring
The severity ladder
Critical
Changes a decision: “I” becomes “we.”Major
Meaning changed; the reader could act wrongly.Minor
Slightly off; the reader still acts correctly.Cosmetic
Style only; meaning intact.
We report the critical-error rate by language, dialect and channel, beside a human baseline.
What you receive: AI test report, evidence package and monitoring plan
Documents your governance board, contracting officer and program office can use without us.
Test report
Findings by language, dialect, channel and severity, with method, limits, human baseline and a unique report ID.Evidence package
For your AI governance board and Chief AI Officer, mapped to M-25-21.Model-card annex
Pashto and Dari results for a vendor model card under M-26-04.Monitoring plan
What to re-test, how often, and what triggers a re-test.Re-test schedule
Dated, and tied to model and vendor updates.
Test report, finding F-03
- Critical errors
3.0%
- Major errors
7.5%
- Word error rate
31%
- Human baseline, critical
0.5%
What the evidence does not establish
- Safety for every use: findings hold for what was tested.
- Future versions: updates call for a re-test.
- Anything about individuals: we assess tools, never people.
Why Ariana Nexus for independent AI testing
No accredited scheme certifies AI for Pashto or Dari, and no public multilingual jailbreak benchmark we reviewed includes them. So we write native test sets and publish our method.
Why us
Native-written, never machine-translated
How we deliver
Our own people, one point of accountability
How we differ
We test the system as it runs
Security, independence and federal registration
Testing runs inside your environment or a boundary you approve, under an NDA signed first.
Ariana Nexus is an independent advisory and professional services firm for Afghan matters in Washington, D.C., founded in 2025, that independently tests government AI systems in Pashto, Dari and 22 more Afghan languages.
Independence wall
Nothing we built, deployed, sold, trained or supplied data for.Conflicts declared
On every report, with any vendor payment.Ethics & Safety Board
Six powers over every engagement:- 01 Approve
- 02 Approve with conditions
- 03 Require changes
- 04 Pause
- 05 Decline
- 06 Stop
Your environment
Testing runs there, or in a boundary you approve.No retention
None kept; none used for training.NDA first
Before any material changes hands.U.S.-located testers
For all U.S. work.Our own people
One point of accountability.
- SAM.gov
Registered
- UEI
M2UDMUDFXFL9
- CAGE
1Z3A3
Who tests: Pashto and Dari linguists and ML evaluators
The firm’s own people: native Pashto and Dari linguists and ML evaluators, alumni and scholars of leading universities, not bilinguals.

Executive oversight
Hassan Ukasha
B.S.
Cornell UniversityM.P.H.
Cornell University

Zeba Haqbani
Senior PartnerB.Sc.
The University of British Columbia
Owns the evaluation harness, scoring pipeline and evidence packaging.
Hussain Ahmad
AnalystM.Eng.
Cornell UniversityPh.D.
University of Chicago
Designs the measurement: metrics, sampling, rater agreement and confidence intervals.
Wasil Peroz
PrincipalB.A.
Milli UniversityM.Sc.
Otto-von-Guericke University Magdeburg
Maps findings to M-25-21, the NIST AI RMF and contract clauses.
Maryam Safi
PrincipalB.A.
Cornell University
Leads public-sector delivery and governance-board evidence packages.
Language-specialist bench
Native linguists who write and score test items.Cultural Compliance Bureau
Reviews every native-script item for variety and register.Cleared-linguist bench
For prime-contractor work that requires cleared personnel.Community connections
Afghan community members, students, alumni and scholars take part with consent, so test sets carry real names, dialects and code-switching.

Languages and where we work
Pashto and Dari in depth; 22 more Afghan languages at the depth available speakers and material allow, confirmed in writing first.
1In depth
- Pashto پښتو
- Dari دری
2Targeted
- Uzbeki
- Turkmeni
- Balochi
- Pashayi
- Aimaq
3By assessment
- The other 17 languages listed below
- Iranian 12
- Pashto پښتو
- Dari دری
- Aimaq
- Balochi
- Ormuri
- Parachi
- Wakhi
- Shughni
- Sanglechi
- Ishkashimi
- Munji
- Yidgha
- Turkic 3
- Uzbeki
- Turkmeni
- Kyrgyz
- Indo-Aryan 3
- Pashayi
- Gawarbati
- Tirahi
- Nuristani 5
- Nuristani (Ashkun group)
- Kati
- Prasun
- Waigali
- Tregami
- Dravidian 1
- Brahui
- French
- German
- Spanish
- Italian
- Hebrew
- Urdu
- Arabic
- Hindi
- Portuguese
- Chinese
- Russian
- Turkish
- Swedish
- Alabama
- Alaska
- Arizona
- Arkansas
- California
- Colorado
- Connecticut
- Delaware
- Florida
- Georgia
- Hawaii
- Idaho
- Illinois
- Indiana
- Iowa
- Kansas
- Kentucky
- Louisiana
- Maine
- Maryland
- Massachusetts
- Michigan
- Minnesota
- Mississippi
- Missouri
- Montana
- Nebraska
- Nevada
- New Hampshire
- New Jersey
- New Mexico
- New York
- North Carolina
- North Dakota
- Ohio
- Oklahoma
- Oregon
- Pennsylvania
- Rhode Island
- South Carolina
- South Dakota
- Tennessee
- Texas
- Utah
- Vermont
- Virginia
- Washington
- West Virginia
- Wisconsin
- Wyoming
- District of Columbia

Questions agencies ask about AI testing in Pashto and Dari
Direct answers for agencies and primes.
What is high-impact AI under OMB M-25-21?
AI whose output is a principal basis for decisions or actions with a legal, material, binding or significant effect on rights, services, safety, critical infrastructure or strategic assets. M-25-21 presumes AI translation high-impact when its output is legally binding or directly informs an agency decision.
How do you test machine translation in Pashto and Dari?
We run native-written, mission-specific items through the system, and qualified translators score each output blind for meaning, severity and variety against a human baseline. Items probe names, kinship terms, ranks, pronouns, dates and code-switching.
Why not use machine-translated test sets?
Because they flatter the system: research finds automatically translated benchmarks fail systematically. Our items are written in Pashto and Dari by native linguists and reviewed by our Cultural Compliance Bureau.
How do you measure accuracy, and what does each metric miss?
Word error rate for speech, chrF for translation trends, COMET only where checked against human scores, and severity-weighted human scoring as the anchor. Automated metrics miss meaning, and in low-resource languages they need human checks.
Are you independent of AI vendors?
Yes. We never test a system we built, deployed, sold, trained or supplied data for. Conflicts, including any vendor payment, are declared on every report and checked by our Ethics & Safety Board.
Do you certify AI systems?
No. No accredited scheme exists for Pashto or Dari AI, so a certificate would claim more than testing can prove. We provide evidence; decisions to deploy, fix, restrict or stop stay with the agency.
What does an AI test report include?
Scope and method, test-set specification, results by language, dialect, channel and severity, a human baseline, security and disparity findings, metric limits and our declared interest, plus an evidence package, model-card annex, monitoring plan and re-test schedule.
Can you test classified or national-security systems?
Only through prime contractors, with personnel holding the clearances each contract requires. NSPM-11 directs NSA and DNI test methods for these systems; we align to them once published. We hold no facility clearance and claim none.
How do agencies buy independent AI testing?
We are registered in SAM.gov (UEI M2UDMUDFXFL9, CAGE 1Z3A3). Agencies buy directly where rules allow, or through prime contractors and BPA holders, including holders of the Chief Digital and AI Office’s independent test and evaluation BPA. We claim no vehicle we do not hold.
How fast can you work?
The method is built for compressed evaluation timelines, and critical findings are reported as soon as they are confirmed. We agree a written schedule at scoping, sized to the system, languages and risk.
How do you protect our data during testing?
Testing runs inside your environment or a boundary you approve, under an NDA signed first. We retain nothing afterward and never train on your data. Testers on U.S. work are in the United States.
Who are the testers?
Native Pashto and Dari linguists working with machine-learning evaluators, alumni and scholars of leading universities. Our Cultural Compliance Bureau reviews every native-script item. Hassan Ukasha, Managing Partner, oversees the program.
Does Azure or Google support Dari?
Partly. Azure AI Translator handles Dari text and Google Translate added Dari in 2024, but as of November 2026 neither Azure AI Speech nor Google Cloud Speech-to-Text lists a Dari speech locale. Systems that route Dari through Iranian Persian drift.
Do you test tools that claim to identify a speaker’s dialect or origin?
Yes, and we report their limits. We make no origin, identity or credibility determinations about individuals. Germany’s migration office stopped applying such software to Pashto in April 2023.
Sources
- OMB M-25-21, Accelerating Federal Use of AI through Innovation, Governance, and Public TrustOffice of Management and Budget, April 2025. Checked 4 November 2026.
- Trump Administration Revamps Guidance on Federal Use and Procurement of AIWiley Rein, April 2025. Checked 4 November 2026.
- OMB M-25-22, Driving Efficient Acquisition of Artificial Intelligence in GovernmentOffice of Management and Budget, April 2025. Checked 4 November 2026.
- OMB memoranda on use and acquisition of AI: what M-25-22 means for government contractorsPilieroMazza, 2025. Checked 4 November 2026.
- OMB M-26-04, Increasing Public Trust in Artificial Intelligence Through Unbiased AI PrinciplesOffice of Management and Budget, December 2025. Checked 4 November 2026.
- White House imposes transparency requirements on contractors (M-26-04)Crowell & Moring, December 2025. Checked 4 November 2026.
- National Security Presidential Memorandum 11 (NSPM-11)GovInfo, June 2026. Checked 4 November 2026.
- Independent test and evaluation BPA, one-page overviewChief Digital and AI Office. Checked 4 November 2026.
- DHS AI use case inventory: USCISU.S. Department of Homeland Security. Checked 4 November 2026.
- 2025 Federal Agency AI Use Case Inventory (consolidated dataset)Office of Management and Budget. Checked 4 November 2026.
- Attorney General memorandum implementing Executive Order 14224U.S. Department of Justice, July 2025. Checked 4 November 2026.
- Executive Order 14434Federal Register, October 2026. Checked 4 November 2026.
- DHS and ICE determinations on high-impact AIAmerican Immigration Council. Checked 4 November 2026.
- AI translation errors and Afghan asylum claimsRest of World, April 2023. Checked 4 November 2026.
- Dialect recognition software (DIAS) and the lawVerfassungsblog, June 2026. Checked 4 November 2026.
- ChatGPT MT: Competitive for High- (but Not Low-) Resource LanguagesRobinson et al., WMT 2023. Checked 4 November 2026.
- PsOCR: Benchmarking Large Multimodal Models for OCR in PashtoHaq, Zhang and Khan, May 2025. Checked 4 November 2026.
- One Language, Three of Its Voices: Evaluating Multilingual LLMs Across Persian, Dari, and TajikiSilkRoadNLP workshop, 2026. Checked 4 November 2026.
- Benchmarking Multilingual Speech Models on PashtoRahman, arXiv, April 2026. Checked 4 November 2026.
- Home Office asylum AI summaries fail checksResultsense, October 2026. Checked 4 November 2026.
- Translation as a Scalable Proxy for Multilingual EvaluationarXiv, January 2026. Checked 4 November 2026.
- Language and voice support for Azure AI SpeechMicrosoft Learn, November 2026. Checked 4 November 2026.
- Speech-to-Text supported languagesGoogle Cloud, 2026. Checked 4 November 2026.
- AI Risk Management FrameworkNational Institute of Standards and Technology. Checked 4 November 2026.
- Review of eleven multilingual safety and jailbreak benchmarksAriana Nexus research, November 2026.
Facts on this page were checked on 4 November 2026.
Start with the system and the mission
Tell us what the system does, who it serves and which languages it meets. We will tell you what can be tested and what the evidence will prove.
Discuss an evaluation