All guides / Scenario
Chinese models in coding agents: test results
Coding agents ask more of a model than chat does: tool calls have to work reliably and come back in the right format. Every row below is a run we actually did: the agent was asked, through XHuoAPI, to create a file (fx also had to run it), and it only counts as passed if the file exists with the right content. Anything we haven't run isn't listed.
Last tested
At a glance
| Tool | API format | Passed | Sometimes failed | Failed |
|---|---|---|---|---|
| Claude Code | Anthropic Messages (/v1/messages) | glm-5.3 deepseek-v4.1-flash-none deepseek-v3.2 qwen3.8-max claude-sonnet-4-6 | deepseek-v4-flash kimi-k3 | — |
| Codex CLI | OpenAI Responses (/v1/responses) | gpt-5.4 gpt-5.5 gpt-4.1 | — | deepseek-v3.2 claude-sonnet-4-6 |
| fx (Vercel Labs) | OpenAI Chat Completions (/v1/chat/completions) | glm-5.3 kimi-k3 | deepseek-v4-flash | qwen3.8-max |
Claude Code Anthropic Messages (/v1/messages)
| Model | Result | Passed / runs | Tool version | Date | $ / 1M in / out |
|---|---|---|---|---|---|
| deepseek-v4-flash | Sometimes failed | 3 / 6 | 2.1.299 / 2.1.302 | $0.35 / $1.06 | |
| glm-5.3 | Passed | 3 / 3 | 2.1.299 / 2.1.302 | $1.12 / $3.52 | |
| deepseek-v4.1-flash-none | Passed | 3 / 3 | 2.1.303 | $0.80 / $3.20 | |
| deepseek-v3.2 | Passed | 1 / 1 | 2.1.299 | $0.23 / $0.34 | |
| qwen3.8-max | Passed | 1 / 1 | 2.1.284 | $1.60 / $4.80 | |
| kimi-k3 | Sometimes failed | 1 / 2 | 2.1.284 | $2.40 / $12.00 | |
| claude-sonnet-4-6 | Passed | 1 / 1 | 2.1.284 | $2.40 / $12.00 |
- deepseek-v4-flash: once, the model replied that the write needed approval and no file was saved; retrying worked.
- kimi-k3: the first run failed with an error; the retry completed.
Codex CLI OpenAI Responses (/v1/responses)
| Model | Result | Passed / runs | Tool version | Date | $ / 1M in / out |
|---|---|---|---|---|---|
| gpt-5.4 | Passed | 3 / 3 | 0.156.1 / 0.162.1 / 0.163.0 | $2.00 / $12.00 | |
| gpt-5.5 | Passed | 1 / 1 | 0.156.1 | $4.00 / $24.00 | |
| gpt-4.1 | Passed | 1 / 1 | 0.156.1 | $1.60 / $6.40 | |
| deepseek-v3.2 | Failed | 0 / 2 | 0.156.1 | $0.23 / $0.34 | |
| claude-sonnet-4-6 | Failed | 0 / 2 | 0.156.1 | $2.40 / $12.00 |
- In Codex CLI, non-GPT models kept showing Reconnecting and then failed. Codex only uses the Responses endpoint, and through XHuoAPI the models that work there are GPT models.
fx (Vercel Labs) OpenAI Chat Completions (/v1/chat/completions)
| Model | Result | Passed / runs | Tool version | Date | $ / 1M in / out |
|---|---|---|---|---|---|
| glm-5.3 | Passed | 3 / 3 | 0.0.13 / 0.0.14 | $1.12 / $3.52 | |
| kimi-k3 | Passed | 3 / 3 | 0.0.13 / 0.0.14 | $2.40 / $12.00 | |
| deepseek-v4-flash | Sometimes failed | 5 / 14 | 0.0.13 / 0.0.14 | $0.35 / $1.06 | |
| qwen3.8-max | Failed | 0 / 5 | 0.0.13 / 0.0.14 | $1.60 / $4.80 |
- deepseek-v4-flash in fx: 5 of 6 runs passed on 10-14, but on 10-18 it kept failing with InvalidFinishReason (on both 0.0.13 and 0.0.14). We traced it to some responses for this model currently sending finish_reason as an empty string, which fx treats as invalid; most clients aren't affected. In fx, use glm-5.3 or kimi-k3 for now.
- qwen3.8-max in fx: every run ended with InconsistentFinishReason. The tool-call fields in the API response are standard, so this looks like a compatibility issue between fx and how this model streams its output.
Setup guide · fx (Vercel Labs)
Cheapest model that passed, per tool
Only models that passed every run, sorted by input price on the standard tier. Prices are XHuoAPI's own and sync with the pricing page.
| Tool | Model | $ / 1M in / out |
|---|---|---|
| Claude Code | deepseek-v3.2 | $0.23 / $0.34 |
| Codex CLI | gpt-4.1 | $1.60 / $6.40 |
| fx (Vercel Labs) | glm-5.3 | $1.12 / $3.52 |
Other coding tools
These tools have setup guides (every UI path checked against the official docs), but we haven't run a full task in them yet, so they aren't in the tables above.
How we test
Each tool is set up exactly as in our guide, with a standard-tier test key, and asked to create hello.py in an empty directory (fx also runs it). When a tool releases a new version we re-test and update the version and date in these tables.