Leaderboard
On-device LLM performance rankings powered by Glicko-2
iPhone 14 Pro Max
iOSRank
#21
Rating
1,921
±16 RD
Win Rate
90.6%
Conservative Rating
1,889
TG Rating
1,888
PP Rating
1,929
Matches
1,086
Record
984W – 102L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| DeepSeek-R1-Distill-Qwen-7B-IQ2_M | 394.30 | 23733.36 | 785.98 | 47463.46 | 2 |
| tinygemma3-Q8_0 | 143.71 | 22221.25 | 143.71 | 22221.25 | 1 |
| MiniCPM4-0.5B.Q2_K | 81.95 | 109.51 | 81.95 | 109.51 | 1 |
| qwen2.5-0.5b-instruct-q8_0 | 62.00 | 1360.96 | 62.63 | 1377.18 | 2 |
| Qwen2.5-Coder-1.5B-Instruct-Q4_K_M | 36.57 | 422.05 | 36.57 | 422.05 | 1 |
| Qwen3-1.7B-Q4_K_M | 35.34 | 373.89 | 35.34 | 373.89 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M | 32.10 | 380.14 | 32.10 | 380.14 | 1 |
| llama-3.2-1b-instruct-q8_0 | 31.90 | 63.94 | 33.26 | 581.02 | 4 |
| Qwen3-1.7B-Q5_K_M | 31.51 | 347.44 | 31.51 | 347.44 | 1 |
| qwen2.5-1.5b-instruct-q3_k_m | 30.81 | 385.69 | 30.81 | 385.69 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M | 28.77 | 43.20 | 28.77 | 43.20 | 1 |
| Qwen2.5-1.5B-Instruct.Q8_0 | 25.66 | 44.74 | 25.66 | 44.74 | 1 |
| qwen2.5-1.5b-instruct-q8_0 | 24.46 | 275.09 | 25.68 | 428.59 | 3 |
| gemma-3-1b-it-Q8_0 | 23.70 | 746.83 | 23.70 | 746.83 | 1 |
| Qwen3-1.7B.Q6_K | 21.59 | 280.60 | 21.59 | 280.60 | 1 |
| Qwen3-1.7B-Q6_K | 19.77 | 28.45 | 19.77 | 28.45 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Fully-Uncensored.i1-Q6_K | 19.53 | 255.03 | 19.53 | 255.03 | 1 |
| gemma-3-1b-it.fp16 | 19.25 | 774.16 | 19.25 | 774.16 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 15.17 | 101.92 | 19.58 | 164.35 | 2 |
| gemma-3-1b-it.Q6_K | 15.09 | 36.87 | 15.09 | 36.87 | 1 |
| Llama-3.2-3B-Instruct.Q6_K | 14.97 | 199.47 | 14.97 | 199.47 | 1 |
| Qwen3.5-0.8B-BF16 | 14.10 | 403.19 | 14.10 | 403.19 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-IQ2_M | 13.82 | 16.03 | 13.82 | 16.03 | 1 |
| Qwen3-4B-Q4_K_M | 13.64 | 140.74 | 13.64 | 140.74 | 1 |
| Qwen3.5-2B-Q3_K_M | 13.09 | 231.41 | 13.09 | 231.41 | 1 |
| Qwen3-4B-Thinking-2507-Q4_K_M | 12.75 | 117.21 | 12.75 | 117.21 | 1 |
| gemma-2-2b-it-Q6_K | 12.30 | 18.19 | 19.44 | 266.96 | 6 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M | 12.08 | 201.29 | 12.08 | 201.29 | 1 |
| gemma-3-4b-it-Q4_K_M | 11.92 | 152.38 | 11.92 | 152.38 | 1 |
| Phi-4-mini-instruct-Q4_K_M | 11.56 | 128.43 | 11.56 | 128.43 | 1 |
| Llama-3.2-3B-Instruct.Q5_K_M | 10.13 | 96.09 | 10.13 | 96.09 | 1 |
| qwen2.5-3b-instruct-q5_k_m | 10.04 | 14.20 | 15.20 | 173.69 | 8 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q5_K_M | 9.98 | 157.67 | 9.98 | 157.67 | 1 |
| Llama-3.2-3B-Instruct-Q6_K | 8.52 | 125.46 | 15.64 | 174.59 | 3 |
| Qwen3-4B-Instruct-2507-UD-Q4_K_XL | 7.36 | 11.46 | 16.11 | 147.00 | 4 |
| Qwen3-4B-Thinking-2507-UD-Q4_K_XL | 7.31 | 91.50 | 7.31 | 91.50 | 1 |
| Qwen3-4B-Instruct-2507-IQ4_NL | 6.84 | 10.45 | 6.84 | 10.45 | 1 |
| gemma-3-4b-it-Q4_K_M | 6.53 | 21.17 | 8.43 | 82.64 | 3 |
| Qwen3.5-4B-UD-Q2_K_XL | 5.87 | 93.52 | 5.87 | 93.52 | 1 |
| DeepSeek-R1-Distill-Qwen-7B-Uncensored.i1-IQ1_S | 5.50 | 61.04 | 5.50 | 61.04 | 1 |
| Llama-3.2-3B-Instruct-IQ3_M | 4.84 | 6.23 | 4.84 | 6.23 | 1 |
| Qwen3-4B.Q4_K_M | 4.82 | 10.56 | 4.82 | 10.56 | 1 |
| Phi-3.5-mini-instruct.Q4_K_M | 2.39 | 12.03 | 2.39 | 12.03 | 1 |
Head-to-Head Record
1–50 of 315 rows
1 / 7
Performance by App Version
ImprovedRegressed