Leaderboard
On-device LLM performance rankings powered by Glicko-2
iPhone 15 Pro
iOSRank
#22
Rating
1,916
±16 RD
Win Rate
90.1%
Conservative Rating
1,884
TG Rating
1,883
PP Rating
1,929
Matches
1,102
Record
993W – 109L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| EXAONE-3.5-7.8B-Instruct-Q4_K_M | 2442.14 | 116609.27 | 2442.14 | 116609.27 | 1 |
| Qwen3-8B-Q4_K_M | 1336.48 | 120381.79 | 1336.48 | 120381.79 | 1 |
| gemma-3-270m-it-Q2_K | 121.30 | 303.37 | 121.30 | 303.37 | 1 |
| google_gemma-3-270m-it-Q6_K | 118.31 | 415.20 | 118.31 | 415.20 | 1 |
| SmolLM2-135M-Instruct-Q2_K | 94.59 | 194.80 | 94.59 | 194.80 | 1 |
| smollm2-360m-instruct-q8_0 | 74.36 | 1708.33 | 74.60 | 1709.75 | 2 |
| Vintern-1B-v3_5-Q8_0 | 69.91 | 169.27 | 69.91 | 169.27 | 1 |
| gemma-3-270m-it-F16 | 69.78 | 4194.50 | 70.07 | 4199.96 | 2 |
| Qwen3-0.6B.Q4_K_M | 64.04 | 109.64 | 64.04 | 109.64 | 1 |
| qwen2.5-0.5b-instruct-q6_k | 62.83 | 1347.45 | 63.76 | 1397.38 | 2 |
| qwen2.5-0.5b-instruct-q4_0 | 62.29 | 111.74 | 62.29 | 111.74 | 1 |
| Qwen3-0.6B-Q6_K | 52.20 | 839.33 | 64.02 | 1037.39 | 2 |
| Meta-Llama-3-8B-Instruct-Q4_K_M | 51.85 | 323.11 | 51.85 | 323.11 | 1 |
| tinyllama-1.1b-chat-v1.0.Q4_0 | 51.22 | 470.34 | 60.67 | 663.53 | 6 |
| Meta-Llama-3-8B-Instruct.Q4_K_M | 45.33 | 455.23 | 57.25 | 599.09 | 2 |
| qwen2.5-1.5b-instruct-q2_k | 41.04 | 452.40 | 41.04 | 452.40 | 1 |
| agentica-org_DeepScaleR-1.5B-Preview-IQ4_XS | 39.28 | 461.17 | 39.28 | 461.17 | 1 |
| QwQ-0.5B-Distilled-SFT-f16 | 37.69 | 1492.78 | 37.69 | 1492.78 | 1 |
| gemma-3-1B-it-QAT-Q4_0 | 37.55 | 356.13 | 40.54 | 657.42 | 2 |
| gemma-3-1b-it-Q8_0 | 35.51 | 829.28 | 35.51 | 829.28 | 1 |
| Qwen3-1.7B-Q4_K_M | 33.12 | 401.81 | 33.12 | 401.81 | 1 |
| llama-3.2-1b-instruct-q8_0 | 31.10 | 200.95 | 32.26 | 647.29 | 4 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q5_K_S | 29.81 | 381.49 | 29.81 | 381.49 | 1 |
| Qwen3-1.7B.Q5_K_M | 29.03 | 315.12 | 29.03 | 315.12 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M | 26.97 | 41.80 | 26.97 | 41.80 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 26.61 | 503.84 | 26.61 | 503.84 | 1 |
| EXAONE-3.5-2.4B-Instruct-Q4_K_M | 25.57 | 172.16 | 25.57 | 172.16 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 25.00 | 488.62 | 25.06 | 530.30 | 2 |
| Deepseek-Qwen2.5-1.5B-Redistil.Q8_0 | 24.64 | 517.92 | 24.64 | 517.92 | 1 |
| SmolVLM2-2.2B-Instruct-Q4_K_M | 23.95 | 34.72 | 23.95 | 34.72 | 1 |
| Qwen3.5-2B-UD-IQ2_XXS | 23.25 | 331.36 | 23.25 | 331.36 | 1 |
| gemma-3n-E2B-it-Q4_0 | 22.59 | 237.19 | 22.59 | 237.19 | 1 |
| qwen2.5-1.5b-instruct-q8_0 | 22.56 | 47.59 | 25.13 | 51.17 | 4 |
| deepseek-r1-distill-qwen-1.5b-q8_0 | 21.60 | 306.96 | 21.60 | 306.96 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q6_K | 21.48 | 31.64 | 21.48 | 31.64 | 1 |
| gemma-3n-E2B-it-Q4_K_M | 21.48 | 221.64 | 21.62 | 223.75 | 3 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 20.91 | 135.75 | 24.76 | 222.27 | 2 |
| gemma-2-2b-it.Q5_K_M | 18.71 | 249.39 | 18.71 | 249.39 | 1 |
| gemma-3-4b-it-q4_0_s | 17.51 | 195.16 | 17.51 | 195.16 | 1 |
| gemma-3n-E2B-it-Q5_K_M | 16.59 | 172.21 | 16.94 | 181.94 | 2 |
| Phi-3.5-mini-instruct.Q4_K_M | 15.68 | 145.00 | 16.54 | 163.83 | 7 |
| SmolLM2-1.7B-Instruct-Q8_0 | 15.34 | 28.42 | 22.22 | 350.61 | 4 |
| Qwen3-4B-Q4_K_M | 14.78 | 141.45 | 16.39 | 160.05 | 2 |
| Phi-4-mini-instruct-Q5_K_M | 14.59 | 185.54 | 14.59 | 185.54 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-f16 | 13.76 | 491.79 | 13.76 | 491.79 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-IQ2_M | 13.75 | 18.85 | 13.75 | 18.85 | 1 |
| agentica-org_DeepScaleR-1.5B-Preview-f16 | 13.75 | 444.58 | 13.75 | 444.58 | 1 |
| gemma-3-4b-it-abliterated-v2.q6_k | 13.64 | 175.21 | 13.64 | 175.21 | 1 |
| gemma-2-2b-it-Q6_K | 13.22 | 22.55 | 19.57 | 306.52 | 11 |
| gemma-3-4b-it.Q4_K_M | 12.30 | 127.64 | 12.30 | 127.64 | 1 |
1–50 of 92 rows
1 / 2
Head-to-Head Record
1–50 of 356 rows
1 / 8
Performance by App Version
ImprovedRegressed