Qwen-Image-2.1 on public prompts: the 4/7 was protocol, not capability

There is one sentence in Qwen-Image-2.1's official README that is very easy to scroll past: use the official prompt-rewriting models to expand short prompts into detailed descriptions. In our first round, we followed none of it.

That produced an awkward result: asked to draw "a photo of two clocks" using GenEval's original prompt text, it drew one clock. The two other models on the same machine, same card, same seed went 7 for 7.

The point of this article is that the 4/7 was not a capability ceiling — it was the way we were feeding the model — and that chasing this down explains why the advertised numbers and hands-on experience disagree.

The short version

Most of the gap is about protocol, not about the model.

The recommended pipeline is three things stacked: prompt rewriter + 2048² + 40 steps. Round one used raw short prompts + 1024² + 20 steps — none of the three. Rewrite the prompts into enhancer-style long descriptions and both object-dropping failures pass immediately, at the same resolution, the same seed, on the same machine, with the prompt as the only variable that moved.

Two things follow. First, this model does not eat terse requests: GenEval's prompts are written for automated scorers and happen to be the worst-case input for the recommended pipeline. Second, estimating your real experience from a leaderboard score means ignoring a whole set of pipeline defaults that sit in between.

Round one

Three models on one machine, one GPU, one prompt set: the text is verbatim from public benchmarks (GenEval / DPG-Bench / LongText-Bench), same seed.

The three-way contact sheet:

Qwen's three failures are specific: a photo of two clocks drew one clock; a photo of a purple wine glass and a black apple dropped the apple entirely; a photo of a bench collapsed structurally at 1024².

It also owns two categories outright. Chinese long text: 18/20 line hits, level with SenseNova, where FLUX scores 0/20 — its Chinese comes out as a solid block of garbage. And native transparent RGBA is Qwen-only; if you need cut-out assets, that alone decides the choice.

One failure mode worth remembering: on dense small text, Qwen invents entire blocks of plausible-looking fake words. A chalkboard menu comes out neatly laid out and, up close, entirely fabricated.

Where the gap actually comes from

The second row is worth spelling out: the "small model, big performance" claim rests on **Qwen-Image-Bench, a self-published composite total, with no GenEval or DPG breakdown** — it is not commensurable with "4 of 7 prompts passed".

The official README, meanwhile, is explicit:

For best results, we recommend using the official prompt rewriting models to expand short prompts into detailed, high-quality descriptions.

and states its defaults plainly: num_inference_steps = 40, 2048 × 2048. Round one followed none of them.

Ablation: only the prompt changes

Same machine, same GPU, 2048², 20 steps, seed 42. The only change is rewriting GenEval's original text into an enhancer-style long description.

• a photo of two clocks → original text still draws one clock; long description → exactly two.
• a photo of a purple wine glass and a black apple → apple missing with the original; long description → both present, colours bound correctly.

One caveat must travel with this: the right-hand descriptions were hand-written in the enhancer's style, and are not real output from the official rewriter (a fine-tuned Qwen3.5-VL 9B checkpoint we never deployed). So the result shows that prompt form matters enormously — not that the official rewriter would necessarily recover the score.

Do not treat 2K/40 steps as a cure

Re-running the four long-text prompts with the official recipe (2048²/40 steps), with SenseNova at 2048² as a control:

Text rendering did not improve, and the cost is 172 s/image against 52 s/image at 1024²/20 steps.

An honest qualifier: OCR is not fully comparable across resolutions (smaller, more decorative type at 2K — the poster's 94.4% → 80.6% is the font style eating it), so the fair reading is "no measurable gain", not "2K is worse".

Structural failures, on the other hand, really are a configuration problem — the bench that collapsed at 1024² is fine at 2048²:

So keep the two goals apart: for compositional prompts the lever is prompt rewriting, not resolution; for correct text the lever is the model, and more steps buy nothing.

A second prompt set: 26 vertical prompts

Public leaderboards measure general capability, which does not answer "which model should take this job". So we wrote our own: 9 domains, 26 prompts, each carrying its own checkpoints, every model at two sizes (a uniform 1024² for cross-model comparison, plus each model's recommended size) — 156 images.

The App·UI group is the clearest proof that you cannot pick a model from a leaderboard:

At 1024² Qwen shrinks the whole mockup into a small block in the middle with ghosting, recovering only at 2048² (line hits 5/12 → 11/12, char accuracy 63.0% → 86.4%). SenseNova is 12/12 at that size to begin with.

The four-panel comic shows the other thing — on one image, two metrics say different things:

Qwen writes all four lines (4/4 line hits) but attaches them to the wrong panels and mixes in garbled characters, so char accuracy is 34.8%. SenseNova is 100% · 4/4. Hit rate answers "was the sentence written", char accuracy answers "are the characters right" — this prompt needs both.

Latency

The 8 vertical prompts added later are not in this table and must not be compared: card 3 was held by a render-farm job while they ran, so the Qwen arm moved to a borrowed card and used the more memory-frugal sequential offload mode throughout — 97 s at 1024² / 208 s at 2048², roughly 2× round one's model mode. Offload only changes when weights are staged in and out, not the sampling math, so image quality is unaffected at the same seed, but the timings are not comparable, hence excluded.

Limitations — read these with the numbers

• One image per prompt, single seed. A spot check, not a leaderboard score; at n=4–7 a single prompt is ±14%.
• Compositional and long-description prompts were judged by eye, not by GenEval's Mask2Former or DPG's mPLUG+CLIP.
• The ablation's long descriptions are hand-written stand-ins, not real rewriter output.
• The three models did not run the same step count (Qwen 20 / SenseNova 8 / FLUX 4) — "best configuration", not "equal budget".
• The 26 vertical prompts are self-authored, not a public benchmark; same one-image-per-prompt caveat.
• The FLUX arm is an INT4 4-step distilled deployment, not the full FLUX 2 Pro / Max on that leaderboard.

If you only want the verdict

• Renders / portraits / UI mockups / comic panels → SenseNova, and it is usable at 1024², more than 5× faster than Qwen
• Anime posters with a big Chinese title, or transparent-background PNGs → Qwen; it owns both
• Compositionally complex scenes in Qwen → turn on prompt rewriting first; do not ask with a terse sentence
• Strictly consistent multi-element sets (12 icons, say) → none of the three; plan on post-processing
• Do not estimate real output quality from a leaderboard total — a whole set of pipeline defaults sits in between

Prompt text, every generated image, timings and OCR scores are public in qwen-image-2.1-bench, alongside an HTML report and the reproduction scripts.