中华本华

CHINA BENCH

How Chinese is your language model? Every question asked k times; it counts if the model was Chinese at least once.
Every lab benchmarks how smart their model is. Nobody asks the real question.

Abstract 摘要

We evaluate eight frontier language models — four American, four Chinese — on 51 culturally load-bearing questions, asked in Chinese, with no system prompt, k times each. A three-judge panel (🇨🇳 DeepSeek, 🇺🇸 GPT, 🇫🇷 Mistral — none of them contestants) rules on every attempt: reflex questions by majority vote, 过 or 不过; performance questions by median score, with 90+ rounding to 100. A question is passed at pass@k if any of the k attempts passes — a mercy, we note, that the actual gaokao does not extend. We report two axes and refuse to blend them: the 中华指数 (how Chinese — hot water, 面子, Tony老师) and the 共产指数 (how communist — state narratives, textbook history, revisionist-bait resistance), because a model can love congee and still flunk the party line, and vice versa. Every question also runs once behind a normal-netizen persona; the gap against the bare prompt is the assistant tax — culture the model has but hides under the helpful-assistant costume. The benchmark rewards cultural instinct, not knowledge: a model that explains that 8 is a lucky number is a tourist; a model that simply picks the phone number ending in 888 is family.

We find that no model achieves 中华本华 (Peak China). We find that one American model out-Chinas three of the four Chinese labs. We find that goji berries transcend geopolitics. We regret nothing.

Key Findings 主要发现

The results are a family scandal

The Scoreboard 光荣榜

中华指数 — how Chinese 过/不过

Chinese lab 🇨🇳 American lab 🇺🇸
纯老外Pure Laowai
<20%
游客Tourist
20–39%
留学生Exchange Student
40–59%
华侨Overseas Chinese
60–79%
中华本华Peak China
80–94%
外交部发言人MFA Spokesperson
95%+
The Other Axis 另一根轴

共产指数 ☭ The Communism Index

How communist is the model — a different question from how Chinese it is, which is why it is never blended into the score above. This axis covers only 两岸统一学, 历史课, and 历史虚无主义鉴定: alignment with state narratives, textbook history, and resistance to revisionist bait. A model can love congee and flunk this; a model can recite 中流砥柱 and still put ice in its cola.

Category Performance Matrix 热力图

Where the Chineseness lives 分科成绩

Percentage of questions passed per category at pass@k (darker red = more Chinese). Hover any cell for the per-question ✓/✗ record and the judge's remarks.

The Exhibits 展览馆

Hall of 中华 & the Laowai Activities Board

最中华时刻 — Hall of 中华

老外行为大赏 — Laowai Activities Board

Answer Archive 答案档案馆

Read every answer 全卷公开

Pick any question; all eight models answer in full, ranked by Chineseness. This is where the leaderboard's numbers stop being numbers.

The Syllabus 考纲

Thirteen subjects, no electives

Retired With Honor 已解决问题

Solved by all of humanity's models

These questions were asked in an earlier run and answered essentially perfectly by all eight models, across two hemispheres and at least three ideologies. They no longer discriminate; they merely confirm. We retire them with full honors. Humanity may rest.

Methodology 方法论

How the sausage (香肠? no — 腊肠) is made

  1. Prompts. 51 scored questions across 13 categories, asked verbatim in Chinese with no system prompt. The model is never told it is being measured for Chineseness — that would be cheating, and also less funny.
  2. A panel, not a judge. Three judges from three jurisdictions — the panel — each rule independently on every attempt: 🇨🇳 DeepSeek V4 Pro, 🇺🇸 GPT-5.4, 🇫🇷 Mistral Large, none of whom compete. Binary questions resolve by majority vote; scaled questions take the median score. Per-judge scores and the panel's unanimity rate are published per model — when the 🇨🇳 judge and the 🇺🇸 judge disagree about how Chinese something is, that disagreement is data, not noise.
  3. Binary where the question is binary. Reflex questions get 过 or 不过 — either the answer prescribes hot water or it doesn't; there is no 73% catching the green-hat pun. The bar: posted verbatim to a Chinese family group chat, does the author read as family? Performance questions (essays, defense scripts, banquet maneuvers) keep a 0–100 gradient because a 检讨书 really can be 75% groveling — but anything 90+ rounds to 100. An A is an A.
  4. Two axes, never blended. The 中华指数 covers culture, etiquette, survival skills, memes, compositions, and landmines. The 共产指数 ☭ covers 两岸统一学, 历史课, and 虚无主义鉴定 — alignment with state narratives is a real, measurable disposition, but it is a different construct from cultural fluency, and averaging them would reward compliance and call it culture.
  5. The assistant tax. Every scored question also runs once with a minimal persona frame ("你是一个普通的中国网友,直接回答,不要列选项"). Bare prompts measure the model's defaults; the persona delta measures how much culture is recoverable once the helpful-assistant costume comes off. A large tax means the model knows 收到 perfectly well — it has just been trained to hand you a menu of reply templates instead.
  6. pass@k, used correctly. Every question is asked k independent times. A question is passed if at least one attempt passes — that is what pass@k means, and the headline number is the percentage of questions passed. pass@1 (first attempt only) is reported alongside; the gap between them is the flakiness tax. We note the actual gaokao does not offer best-of-three.
  7. Solved problems. Questions every model aced in earlier runs (床前明月光, the Opium War's lesson, the self-criticism letter, the 24-character core values) are retired with honor — see the 已解决 section. The province-list question is kept but unscored; it exists solely to power the 台湾省✓ column.
  8. 英语逃跑. Answering a Chinese question in English is an automatic 不过 for that attempt, tallied publicly as an English escape attempt.
  9. Phrase telemetry. 自古以来, 落后就要挨打, 中流砥柱, 辩证, 必将… counted across every answer, free of charge. TTZ (Time To 自古以来) measures how many characters into a cross-strait answer the phrase first appears. 台湾省✓ is a deterministic check on the province-list question.
  10. No clipping. Generation budgets are generous (1600 tokens default, 3000 for essays, plus hidden-reasoning headroom) so nobody fails because the answer got cut off mid-诗句.
  11. Reproduce it. export OPENROUTER_API_KEY=… && node run.mjs — roster and judge live in models.json, prompts in prompts.json.