GPT-6-Luna
OpenAICodexReasoning: high
GPT-6-Luna gets about 2 vocabulary questions in 10 right. It does best in Kabyle. Its weakest spot is writing Tarifit words in Tifinagh. It swaps Tifinagh and Latin letters well (88.3 out of 100).
Rank 4 of 9
TifinaghBench score
18.2out of 100
95% interval 17.3–19.0
17.7 with strict spelling
0–100, with 95% interval
Script score
88.3out of 100
95% interval 87.2–89.4
70.8 with strict spelling
Transliteration score, 0–100
Scores by dialect and task
Share of the score earned in each cell, 0–100. The transliteration column measures the script, not the vocabulary.
Standard Moroccan Amazigh
- Tifinagh → English
- 16.1118 correct, 85 partial, 797 wrong of 1000
- English → Tifinagh
- 3.535 correct, 0 partial, 965 wrong of 1000
- Tifinagh ↔ Latin
- 94.9949 correct, 0 partial, 51 wrong of 1000
Tarifit
- Tifinagh → English
- 19.1142 correct, 96 partial, 755 wrong of 993
- English → Tifinagh
- 2.727 correct, 0 partial, 966 wrong of 993
- Tifinagh ↔ Latin
- 75.6756 correct, 0 partial, 244 wrong of 1000
Kabyle
- Tifinagh → English
- 49.5428 correct, 131 partial, 438 wrong of 997
- English → Tifinagh
- 18.2181 correct, 0 partial, 816 wrong of 997
- Tifinagh ↔ Latin
- 94.4944 correct, 0 partial, 56 wrong of 1000
| Tifinagh → English | English → Tifinagh | Tifinagh ↔ Latin | |
|---|---|---|---|
| Standard Moroccan Amazigh | 16.1118 correct, 85 partial, 797 wrong of 1000 | 3.535 correct, 0 partial, 965 wrong of 1000 | 94.9949 correct, 0 partial, 51 wrong of 1000 |
| Tarifit | 19.1142 correct, 96 partial, 755 wrong of 993 | 2.727 correct, 0 partial, 966 wrong of 993 | 75.6756 correct, 0 partial, 244 wrong of 1000 |
| Kabyle | 49.5428 correct, 131 partial, 438 wrong of 997 | 18.2181 correct, 0 partial, 816 wrong of 997 | 94.4944 correct, 0 partial, 56 wrong of 1000 |
By dialect
Meaning tasks only
- Standard Moroccan Amazigh9.8
- Tarifit10.9
- Kabyle33.8
By task
- Tifinagh → English28.2
- English → Tifinagh8.1
- Tifinagh ↔ Latin88.3
Tifinagh → English grades
Free-text meanings, graded by the judge or matched exactly.
- Correct
- Partial
- Wrong
- Failed
Standard Moroccan Amazigh118 correct, 85 partial, 797 wrong of 1000
Tarifit142 correct, 96 partial, 755 wrong of 993
Kabyle428 correct, 131 partial, 438 wrong of 997
Character error rate
For exact-match answers: the smallest edit distance to an accepted answer, divided by its length. Lower is better; it shows near misses that a 0 score hides.
| English → Tifinagh | Tifinagh ↔ Latin | |
|---|---|---|
| Standard Moroccan Amazigh | 86.6% | 1.1% |
| Tarifit | 94.4% | 24.2% |
| Kabyle | 64.8% | 1.4% |
The run
- Setup
- Official setup
- Harness
- Codex 0.163.0@openai/codex-sdk 0.163.0
- Judge
- Claude Haiku 5.5 (Claude Code)
- Sample
- 1,000 words per dialect, seed 1
- Batch size
- 100
- Sessions
- 93 answer, 30 judge
- Tokens
- 1,003,026 in · 282,988 out · 126,226 reasoning
- Model cost
- Not reported by the harness
- Judge cost
- $0.20
- Dates
- 19 Oct 2026, 20:44 UTC → 19 Oct 2026, 21:04 UTC
- CLI version
- 0.1.0
Dataset checksums
SHA-256 of each dictionary file, so the run can be checked once the datasets are released.
- Standard Moroccan Amazigh
633505d9ccd2244f9a53b64f56bd153c428cc3dbd7229066845404bf3d7c5060 - Tarifit
d144a5619aa31cc06ffcc4eb26c361c65ca2c71451dfdfbea7fa134206481c4f - Kabyle
ed3a9882ac0f4ef10630917a302fd7d9ff6ac706654bb317b94c715f6b4cf9ea