integration bench
rev 01 · mean task score · public task setintegration bench
rev 01 · mean task score · public task set| # | Model | Pass 1 | Resolved |
|---|---|---|---|
| 1 | 
Claude Fable 5.1xhigh | 69.35±4.38 | 70% |
| 2 |
Claude Fable 5xhigh | 67.68±4.72 | 68% |
| 3 | 
GPT-6 Astraxhigh | 65.40±5.12 | 66% |
| 4 |
GPT-5.6 Solxhigh | 63.62±5.14 | 64% |
| 5 | 
DeepSeek V4 Pro (max)max | 63.35±4.91 | 64% |
| 6 | 
Tencent HY 4 Previewmax | 62.62±5.36 | 64% |
| 7 | 
Qwen 3.8 2.4T A95Bdefault | 61.68±4.57 | 62% |
| 8 | 
Grok 4.6default | 61.18±5.48 | 62% |
| 9 | 
Gemini 3.8 Flashhigh | 59.68±4.83 | 60% |
| 10 | 
GLM 5.3 Flashmax | 59.37±5.63 | 60% |
| 11 |
Claude Opus 5xhigh | 59.32±5.07 | 60% |
| 12 |
Qwen 3.8 27B (xhigh)xhigh | 57.73±4.46 | 58% |
| 13 | 
Muse Spark 1.3high | 57.66±5.22 | 58% |
| 14 | 
Kimi K3default | 57.47±4.96 | 58% |
| 15 |
GLM 5.3max | 57.31±5.41 | 58% |
| 16 |
DeepSeek V4 Flash (max)max | 55.68±4.68 | 56% |
| 17 |
Claude Sonnet 5xhigh | 53.85±4.88 | 54% |
| 18 |
GPT-5.6 Terraxhigh | 53.79±5.33 | 54% |
| 19 |
Muse Spark 1.2default | 53.70±4.54 | 54% |
| 20 |
GPT-5.6 Lunaxhigh | 51.78±5.58 | 52% |
| 21 |
GLM 5.2 (xhigh)xhigh | 49.68±5.86 | 50% |
This is the short version. Every number above comes from a recorded run. 1050 of them are published, each with every tool call, every vendor request and the diff.
Replay them at integrationbench.com →