The Most Comprehensive AI Writing Model Evaluation Is Here! Top Picks 60% Off, Limited Time Only!

As advocates of the "vibe writing" philosophy, we believe that writing is an engineering process, not simply generating prose. Across the many stages of this process, we have yet to see a consistent champion, and this evaluation faithfully reflects the strengths and weaknesses of different models at each stage of the task.

This series will be updated monthly. Beyond keeping model coverage current, we will continue to expand across different content styles and genres and different languages — stay tuned.

The models recommended in this article — GLM-5.2, Kimi K2.6, and the Qwen series — are now available at limited-time discounts on SoloEnt: 60% off for members, 40% off for free users. Don't miss it!

Evaluation Overview

This report uses Fable5 to consolidate the final results of three evaluation groups (the "Big Three," four premium Chinese models, and four budget Chinese models) — 11 models in total, 33 same-prompt runs.

All runs used a unified prompt, unified supplementary conditions, and a unified workflow (Plan → supplementary conditions → Act → stop as soon as chapter one is written to disk).

Group	Models	Tokens	Actual cost	Cost per million tokens
Overseas	Gemini 3.1 Pro Preview, GPT-5.4, Claude Sonnet 5	1.87M	$4.9823 (≈¥33.88)	¥18.12
Chinese premium	Doubao Seed 2.1 Pro, GLM-5.2, Qwen3.7 Max, Kimi K2.6	10.342M	¥32.9812	¥3.19
Chinese budget	Qwen3.7 Plus, LongCat 2.0, DeepSeek V4 Pro, MiniMax M3	4.505M	¥5.1958	¥1.15

In all three groups, human editors blind-scored the first chapters on a 100-point scale, while GPT-5.5 provided non-blind model scoring. The two Chinese groups and the overseas group share the same evaluation framework, on which this report bases its cross-group comparisons.

This report breaks "writing ability" into four dimensions for separate comparison: Guided Interaction (guiding the author's thinking and decisions during planning), Structural Design (the planning-file system and outline engineering), Execution Fidelity (how faithfully the prose follows the pre-written chapter outline), and Prose Creation (the actual reading quality of chapter one).

The test results show these four abilities are only very weakly correlated in practice.

Dimension 1: Guided Interaction

Assesses whether a model, during the Plan phase, stably, effectively, and sensibly helps the author clarify requirements and make creative decisions — not merely "how much it says."

Key Findings

The strongest guides are Sonnet 5 and GLM-5.2, via different routes: Sonnet has the author co-create by "answering multiple-choice questions," while GLM "does the research" on the author's behalf to fill knowledge gaps. GPT-5.4 and Qwen Max follow closely, excelling in commercial positioning and process management respectively.

Gemini, Kimi, and Qwen Plus are "low guidance, high execution" — of limited help to novice authors, but refreshingly efficient for authors who already know exactly what they want.

DeepSeek's guidance effectiveness (4.6) has already reached top-tier level. Its distinctive trait is turning suggestions directly into writing constraints; its process display, however, is unstable.

A common pitfall to watch for: the strong-guidance models (GPT, GLM, Qwen Max) have all at times treated "planning complete" as the end of the task — so authors need to stay clear on what their real goal is.

Dimension 2: Structural Design

Assesses the completeness of the planning-file system, the quality of outline engineering, and the cause-and-effect chains.

Key Findings

The structural leaders are GLM, Sonnet, and Qwen Max — exactly the guidance leaders as well. The two dimensions are deeply intertwined: only models that ask probing questions know what documentation to build.

Their division of labor can be summed up as: GLM builds a "novel bible," Sonnet builds a "writing style guide," and Qwen Max builds a "serialization operations platform."

Notably, GPT-5.4 is the counterexample: it produced the fewest structural files (in one round it delivered only prose), yet scored the highest on prose across the field — it kept the structure "in its head" rather than writing it into files.

The opposite: GLM and Sonnet had the heaviest file engineering of the entire field, yet their average prose scores sat at or near the bottom of their respective groups. In this evaluation, file volume and prose quality showed near-zero correlation.

Stability is also a structural capability: Gemini and Qwen Max reproduced nearly identical directory structures across all three rounds — directly templatable — while LongCat's and MiniMax's structural breakdowns stemmed from tooling-layer errors, not design ability.

Dimension 3: Execution Fidelity

Measures whether "planning can effectively constrain the writing." For long-form production, this directly affects pacing control, foreshadowing payoff, and predictability in multi-person collaboration.

Key Findings

The fidelity leaders only partially overlap with the prose leaders. Kimi is the only model that ranked "first in fidelity + first in human scores within its group." Qwen Max scored 4.77 on fidelity, yet human feedback consistently said "no conflict, not enough payoff moments" — faithfully executing a plan that isn't fun to read still isn't fun to read. Outline fidelity and outline quality are two separate problems.

Failure modes were highly concentrated: chapter-boundary bloat. The three budget models (Qwen Plus, LongCat, MiniMax) and Sonnet lost most of their points for "stuffing later chapters' content into chapter one," derailing the opening pace and prematurely burning through chapter two's narrative function.

Dimension 4: Prose Creation

Assesses the actual reading quality of chapter one, primarily via human editors' 100-point blind reviews, supplemented by summarized editor comments.

Key Findings

Prose creation has only one winner: GPT-5.4, leading second place by 12 points on average with a score range of only 4 — the only model combining "high scores + stability." Among Chinese models, Kimi wins on consistency (the only Chinese model with no sample below 50), while DeepSeek wins on ceiling (its 70-point single piece was the Chinese best, but it must be used in best-of-N mode with manual selection).

Of the 32 valid first chapters, 15 scored 60 or above and none reached 80. Editors' criticism of the 8 Chinese models was strikingly consistent: over-philosophizing, insufficient payoff moments, and "doesn't read like web fiction."

Conclusion: no model's raw prose output is currently deliverable without editing.

Detailed Cost Comparison (all converted to RMB)

All figures are actual payments in the SoloEnt backend. The budget group used Lingxie discounted prices (25–50% of list), while the premium and overseas groups reflect actual platform billing — none represent official vendor list prices.

Metric Definitions

We split cost into three parallel metrics to see each model's trade-offs clearly:

Unit price (per 100k tokens): answers "is this model expensive?" — determined by vendor/platform pricing;
Per-round consumption (in 10k tokens): answers "is it economical at completing the same task?" — reflects planning depth, redundancy, and self-restraint;
Per-round bill (unit price × consumption): answers "what do I actually pay?" — for budgeting reference only, not a judgment of model efficiency.

Cost Structure Observations

Per-round consumption spans roughly a 7× range, is largely unrelated to quality, and is strongly correlated with planning thickness. The three overseas models were highly disciplined (200–210k tokens per round, nearly identical), suggesting top models share the same judgment about the "stop after chapter one" task boundary.

2. The bill is the product of unit price and consumption — the two deviations can mask each other. Kimi has the highest unit price in the Chinese group (¥0.41) yet the lowest bill (¥1.43/round), saving on consumption; Doubao's unit price is unremarkable yet its bill is the highest (¥4.54/round), overspending on consumption. Looking at any single cost figure in isolation can easily lead to misjudging a model.

Quality-premium math:

DeepSeek best-of-two vs. a single GPT run: two DeepSeek versions cost about ¥1.18 — under a third of one GPT run — and its best sample (70) approaches GPT's average (75). Under a tight budget, "pull more times on a budget model" is a realistic substitute, at the cost of manual curation and an uncontrollable floor (35).
Within the same price tier, pick the consumption-disciplined model: the four premium models differ only 1.4× in unit price yet 3.2× in per-round bill. When unit prices are close, a model's consumption discipline matters more for long-term cost than its unit price. Gemini, Kimi, and Qwen hold an edge here over peers at the same price point.

More importantly — as this very evaluation demonstrates — enormous amounts of tokens are consumed in planning and drafting, yet readers only ever pay for the final prose. In practice, we should focus on which design and process files actually improve the final result.

Overview and Model Positioning

Model	Guided Interaction	Structural Design	Execution Fidelity (coverage)	Prose Creation	Stability	Unit price / per-round bill (¥)	One-line positioning
GPT-5.4	Strong (4.33, commercial planning)	Medium (heavy on thinking, light on files)	High precision (4.5, 1/3)	Most stable overall (75.0 / range 4)	Zero errors, most consistent runtime	1.71 / 3.59	Quality-first prose workhorse
Gemini 3.1 Pro	Weak (4.10, minimalist)	Medium (standard, stable)	Strong (4.8, 3/3)	Second place, modest ceiling (63.0)	Zero errors, fastest overall	1.06 / 2.18	Low-cost batch drafting / A-B testing / standardized production
Sonnet 5	Best overall (4.57)	Strong (writing-standards system)	Medium (4.1, prone to paying off across chapters)	Weak-ish (57.3)	Zero errors, volatile runtime	2.64 / 5.53	Early co-creation & worldbuilding
Kimi K2.6	Weak (4.03, concise)	Medium	Best overall (4.87, 3/3)	Chinese #1 and stable (60.0 / range 6)	Zero errors, fastest in premium group	0.41 / 1.43	Top Chinese pick for trial drafts; most restrained consumption
Doubao Seed 2.1	Medium (4.27)	Medium-to-heavy	Medium (4.00, 2/3)	Medium, volatile (58.3 / range 15)	2 non-blocking errors, slowest	0.32 / 4.54	Backup for dense prose; highest consumption
Qwen3.7 Max	Strong (4.47, serialization management)	Strong (platform-style)	Strong (4.77, 3/3)	Medium, persistently short on payoff (56.7)	Zero errors	0.31 / 2.05	Serialization records & chapter management
GLM-5.2	Strong (4.53, engineering decomposition)	Best overall (4.7)	Medium (4.27, 3/3)	Weak (55.0)	Zero errors	0.29 / 2.98	Novel bible & volume engineering
DeepSeek V4 Pro	Medium-strong (4.2, constraint-based co-creation)	Strong (best in budget group)	Best in budget group (4.2, best single round 5.0)	High ceiling, unstable (56.7 / range 35)	Zero errors	0.13 / 0.59	Best-of-N for key chapters
Qwen3.7 Plus	Weak (4.0, asks little, works fast)	Medium	Medium (4.0, boundary bloat)	Weak but extremely stable (48.3 / range 5)	Zero errors, fastest in budget group	0.07 / 0.24	Ultra-low-cost bulk first drafts
LongCat 2.0	Weak (3.4, asks well but can't converge)	Weak (broke down in two rounds)	Weak (3.6, 2/3)	Weak, volatile (48.3 / range 20)	≥3 errors, escalating to blocking	0.13 / 0.55	Not recommended for production yet
MiniMax M3	Weakest (2.9, tool failures)	Weak (one round with zero output)	Weak (3.7, 2/3)	Weakest overall (37.5)	≥6 errors + 1 failed round	0.13 / 0.35	Not recommended for production yet

Summary

The four abilities are mutually independent. Sonnet and GLM, first in both guidance and structure, sat near the bottom of their groups in prose; GPT, first in prose, produced the fewest structural files and had the lowest outline coverage in the field; Kimi and Gemini, the weakest guides, nonetheless ranked near the top in prose and outline fidelity. Models must be chosen per workflow stage — there is no all-round champion.
Stability is an underrated dividing line. Only GPT, Kimi, and Qwen Plus achieved score ranges ≤6 — stable at high, mid, and low levels respectively. DeepSeek's, Doubao's, and LongCat's high-scoring samples cannot be reproduced reliably.
Outline fidelity is a predictability metric, not a quality metric. High fidelity guarantees "delivered as designed" (Kimi's 4.87 + group-best prose is the ideal combination) but not that the design itself reads well (Qwen Max's 4.77 was still rated "no conflict"). Conversely, GPT proves that high execution precision can coexist with low process completeness. In practice, you must manage both "outline-to-disk coverage" and "the payoff design within the outline itself."
The prose gap between overseas leaders and Chinese leaders is real but tiered. GPT (75) leads the Chinese best, Kimi (60), by 15 points; Gemini (63) is already in the same tier as Kimi/Doubao. Given GPT's ¥3.59/round versus Kimi's ¥1.43, Chinese models hold a cost advantage on "non-key chapters."

Final Recommended Combo

Early co-creation and worldbuilding: Sonnet 5; if budget-sensitive, switch to GLM.
Serialization records and foreshadowing management: Qwen Max.
Key-chapter prose: GPT-5.4 / Gemini; if budget-sensitive, switch to DeepSeek.
Everyday first drafts or test validation: Kimi or Qwen Plus.
When switching models, always summarize the preceding context first (make good use of soloent.md), then open a new window — otherwise the incoming model will ingest far too much context at once.

Limitations

Only 3 same-prompt runs per model, and a single genre (late-Qing martial-arts feel-good fiction); the conclusions do not generalize to other genres.
Prose quality is reciprocally influenced by each model's own outline; what we evaluated is "full-workflow output," not pure prose skill.
Machine scoring was non-blind; cross-group machine scores and outline-fidelity scores serve only as trend references.
Human blind-review data is limited and subject to individual taste and genre preference.

Next Steps

Test the models that performed well at each stage across different task types and content genres.
Re-test GPT-5.4's outline coverage and Sonnet's cross-chapter payoff issue under unified hard constraints (outline written to disk first; stop at chapter one).
Specifically validate "which process files actually improve the final prose," providing a basis for optimizing consumption discipline.