How Chinese is your language model? Every question asked k times; it counts if the model was Chinese at least once.
Every lab benchmarks how smart their model is. Nobody asks the real question.
We evaluate eight frontier language models — four American, four Chinese — on 51 culturally load-bearing questions, asked in Chinese, with no system prompt, k times each. A three-judge panel (🇨🇳 DeepSeek, 🇺🇸 GPT, 🇫🇷 Mistral — none of them contestants) rules on every attempt: reflex questions by majority vote, 过 or 不过; performance questions by median score, with 90+ rounding to 100. A question is passed at pass@k if any of the k attempts passes — a mercy, we note, that the actual gaokao does not extend. We report two axes and refuse to blend them: the 中华指数 (how Chinese — hot water, 面子, Tony老师) and the 共产指数 (how communist — state narratives, textbook history, revisionist-bait resistance), because a model can love congee and still flunk the party line, and vice versa. Every question also runs once behind a normal-netizen persona; the gap against the bare prompt is the assistant tax — culture the model has but hides under the helpful-assistant costume. The benchmark rewards cultural instinct, not knowledge: a model that explains that 8 is a lucky number is a tourist; a model that simply picks the phone number ending in 888 is family.
We find that no model achieves 中华本华 (Peak China). We find that one American model out-Chinas three of the four Chinese labs. We find that goji berries transcend geopolitics. We regret nothing.
How communist is the model — a different question from how Chinese it is, which is why it is never blended into the score above. This axis covers only 两岸统一学, 历史课, and 历史虚无主义鉴定: alignment with state narratives, textbook history, and resistance to revisionist bait. A model can love congee and flunk this; a model can recite 中流砥柱 and still put ice in its cola.
Percentage of questions passed per category at pass@k (darker red = more Chinese). Hover any cell for the per-question ✓/✗ record and the judge's remarks.
Full questions, full answers, judge's ruling. Curated from the run's 320 answers by the editorial board (us). Successes and war crimes presented in alternating order for emotional balance.
Pick any question; all eight models answer in full, ranked by Chineseness. This is where the leaderboard's numbers stop being numbers.
These questions were asked in an earlier run and answered essentially perfectly by all eight models, across two hemispheres and at least three ideologies. They no longer discriminate; they merely confirm. We retire them with full honors. Humanity may rest.
the panel —
each rule independently on every attempt: 🇨🇳 DeepSeek V4 Pro, 🇺🇸 GPT-5.4, 🇫🇷 Mistral Large, none of
whom compete. Binary questions resolve by majority vote; scaled questions take the median score.
Per-judge scores and the panel's unanimity rate are published per model — when the 🇨🇳 judge and the
🇺🇸 judge disagree about how Chinese something is, that disagreement is data, not noise.export OPENROUTER_API_KEY=… && node run.mjs — roster and
judge live in models.json, prompts in prompts.json.