What actually won
Wan 3.0 took the top of the AA-Video-T2V v2.0 leaderboard with an Elo of 1157 and first place in 10 of 20 category rankings. The reported reference price is around twelve dollars per minute of output.
That is a genuine result on the most methodologically careful public benchmark in the space. It is also a narrower claim than “Wan is the best video model”, and the gap between those two statements is where most bad purchasing decisions get made.
How the benchmark is built
The methodology is worth understanding because it determines what the number means.
- Roughly 68,000 human preference votes collected across about 1,000 prompts.
- Blind comparison: voters see two clips from the same prompt without knowing which model produced which.
- Every model is judged at 1080p, so resolution differences do not distort the result.
- Elo scoring aggregates pairwise wins into a single ranking, with category breakdowns underneath.
Why blind matters: brand recognition contaminates open voting badly. Blind pairwise comparison is the only practical way to measure perceived output quality rather than reputation.
What a preference ranking can tell you
Used correctly, this ranking answers one question well: given the same prompt, which clip would a human rather watch. That is a real and useful signal.
- Overall aesthetic quality of a single generated clip.
- How often a model produces something watchable rather than broken.
- Relative standing on broad prompt categories — the 20 sub-rankings are more actionable than the headline number.
The category breakdown is the underrated part. A model that tops the aggregate but loses badly in the category you care about is the wrong choice for you, and the aggregate alone would never reveal that.
What it cannot tell you
The gaps are structural, not flaws in execution.
| Question | Does the benchmark answer it? | What to do instead |
|---|---|---|
| Does it keep a character consistent across shots? | No — clips are judged independently | Run a multi-shot consistency test yourself |
| Does it follow long, specific prompts? | Partially — prompts are short and generic | Test with your real prompt templates |
| Is it fast enough for my workflow? | No | Measure end-to-end latency including queueing |
| Is it affordable at my volume? | No | Compute cost per finished minute, including regeneration waste |
| Does it handle my licence and region? | No | Check terms and availability directly |
The most common mistake: buying on aggregate rank for a workflow that depends on multi-shot consistency. Benchmarks judge single clips; production work almost never ships single clips.
The cost column matters more than the ranking
At production volume, the difference between the top three models on perceived quality is usually smaller than the difference in what you pay per finished minute. Cost has two components people forget.
- Stated price per second of output — easy to compare and usually the only number in the marketing.
- Regeneration waste — the clips you throw away. A cheaper model with a lower hit rate can cost more per usable minute than an expensive one you get right first time.
Measure cost per usable minute, not cost per generated minute. That single change in accounting frequently reverses the ranking.
How to choose for your use case
-
Find your category
Go to the sub-ranking that matches your content type rather than the aggregate. If you make product clips, the category relevant to product shots matters more than overall Elo.
-
Shortlist three, not one
Take the top few in your category. Differences at the top are usually within noise for any individual prompt.
-
Test on twenty of your own prompts
Use prompts you actually ship with, including the awkward ones. Generic benchmark prompts flatter models in ways your specific use case will not.
-
Score for usability, not beauty
Count how many clips you would actually publish. That hit rate, combined with price, decides your real cost.
-
Re-check quarterly
Leaderboard positions move fast. A model that was clearly best in one quarter is often mid-pack the next.
Running your own evaluation
A useful internal benchmark takes about an afternoon and beats reading leaderboards for your specific case.
- Fix twenty prompts that represent real work, including three you expect to fail.
- Generate each on each shortlisted model with identical settings.
- Score blind — shuffle the outputs so you do not know which model produced which.
- Record both a quality score and a pass/fail for publishability.
- Multiply price per minute by the inverse of your pass rate to get cost per usable minute.
Keep the set frozen: once you have twenty reference prompts, never change them. Only a fixed set can detect that a model update quietly made things worse for you.