Leaderboard
On-device LLM performance rankings powered by Glicko-2
iPhone 15 Pro Max
iOSRank
#18
Rating
1,931
±15 RD
Win Rate
91.5%
Conservative Rating
1,901
TG Rating
1,887
PP Rating
1,945
Matches
1,241
Record
1136W – 105L
Models Tested
| Model | TG Median (tok/s) | PP Median (tok/s) | TG Best | PP Best | Runs |
|---|---|---|---|---|---|
| llama-3.1-8b-instruct-q4_0 | 2113.56 | 94180.11 | 2113.56 | 94180.11 | 1 |
| DeepSeek-R1-Distill-Qwen-14B-IQ2_XS | 1179.45 | 37109.39 | 1179.45 | 37109.39 | 1 |
| DeepSeek-R1-Distill-Qwen-14B-IQ2_S | 284.22 | 20905.89 | 284.22 | 20905.89 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-uncensored.f16 | 120.09 | 11392.49 | 226.44 | 22307.36 | 2 |
| ruvltra-claude-code-0.5b-q4_k_m | 95.93 | 1298.87 | 95.93 | 1298.87 | 1 |
| gemma-3-270m-it-F16 | 76.14 | 306.02 | 76.14 | 306.02 | 1 |
| Mistral-Nemo-Instruct-2407-IQ2_M | 61.59 | 523.07 | 61.59 | 523.07 | 1 |
| Nano-Vlm-Processor-494M-F16 | 43.80 | 1690.53 | 43.80 | 1690.53 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q2_K | 38.75 | 425.00 | 38.75 | 425.00 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-uncensored.Q4_K_M | 35.56 | 430.59 | 35.56 | 430.59 | 1 |
| gemma-3-1b-it.Q8_0 | 34.73 | 807.13 | 34.73 | 807.13 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-IQ2_M | 32.54 | 338.02 | 41.01 | 447.09 | 2 |
| Qwen3-VL-2B-Instruct-Q4_K_M | 31.64 | 461.32 | 31.64 | 461.32 | 1 |
| Llama-3.2-1B-Instruct.Q8_0 | 29.94 | 650.51 | 29.94 | 650.51 | 1 |
| llama-3.2-1b-instruct-q8_0 | 29.27 | 59.88 | 29.27 | 59.88 | 1 |
| Qwen2.5-0.5B-Instruct-Q4_K_S | 29.06 | 79.51 | 29.06 | 79.51 | 1 |
| qwen2.5-1.5b-instruct-q8_0 | 25.80 | 490.55 | 26.38 | 550.99 | 4 |
| Qwen3-1.7B.Q4_K_M | 25.54 | 219.45 | 32.77 | 404.57 | 2 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 24.54 | 473.85 | 24.54 | 473.85 | 1 |
| tinyswallow-1.5b-instruct-q5_k_m | 24.52 | 30.54 | 24.52 | 30.54 | 1 |
| Dolphin3.0-Qwen2.5-3b-IQ2_M | 23.98 | 202.63 | 23.98 | 202.63 | 1 |
| tinyswallow-1.5b-instruct-q8_0 | 23.71 | 326.38 | 24.58 | 456.84 | 6 |
| DeepSeek-R1-Distill-Qwen-1.5B-uncensored.Q2_K | 22.27 | 31.96 | 22.27 | 31.96 | 1 |
| qwen2.5-3b-instruct-q4_k_m | 20.75 | 216.24 | 20.75 | 216.24 | 1 |
| lucy_128k-Q8_0 | 20.52 | 41.69 | 20.52 | 41.69 | 1 |
| gemma-3-1b-it.fp16 | 20.24 | 865.73 | 20.24 | 865.73 | 1 |
| google_gemma-3-1b-it-bf16 | 20.20 | 620.88 | 20.20 | 620.88 | 1 |
| Qwen3.5-2B-Q4_K_M | 19.28 | 314.75 | 19.28 | 314.75 | 1 |
| DeepSeek-R1-Distill-Qwen-1.5B-Q8_0 | 19.00 | 244.99 | 24.63 | 459.78 | 4 |
| Gemmasutra-Mini-2B-v1-Q6_K | 18.64 | 254.87 | 19.28 | 310.21 | 4 |
| Qwen3.5-2B-UD-Q4_K_XL | 18.59 | 297.26 | 18.59 | 297.26 | 1 |
| SmolLM2-1.7B-Instruct-Q8_0 | 17.41 | 31.07 | 17.41 | 31.07 | 1 |
| sarashina2.2-3b-instruct-v0.1-q4_k_m-imat | 16.40 | 150.46 | 16.40 | 150.46 | 1 |
| gemma-3-4b-it-Q4_K_M | 16.10 | 188.87 | 16.10 | 188.87 | 1 |
| qwen2.5-3b-instruct-q5_k_m | 15.02 | 174.74 | 127.96 | 22651.11 | 16 |
| google.gemma-3-4b-it.Q6_K | 13.14 | 173.09 | 13.14 | 173.09 | 1 |
| mistral-7b-uncensored-Q2_K | 13.10 | 83.49 | 13.29 | 83.63 | 2 |
| gemma-3n-E2B-it-IQ4_NL | 12.75 | 21.40 | 12.75 | 21.40 | 1 |
| gemma-3n-E2B-it-Q5_K_M | 12.61 | 21.43 | 12.61 | 21.43 | 1 |
| nsfw-3b-q4_k_m | 12.29 | 19.02 | 12.29 | 19.02 | 1 |
| Llama-3.2-3B-Instruct-Q6_K | 12.25 | 160.59 | 16.08 | 218.46 | 12 |
| Llama-3.2-3B-Instruct.Q8_0 | 12.24 | 204.47 | 12.24 | 204.47 | 1 |
| gemma-2-2b-it-Q6_K | 11.95 | 61.15 | 19.71 | 307.76 | 16 |
| Qwen_Qwen3-4B-Thinking-2507-IQ4_NL | 11.56 | 116.22 | 11.56 | 116.22 | 1 |
| translategemma-4b-it-Q6_K | 11.25 | 165.01 | 11.25 | 165.01 | 1 |
| Phi-4-mini-instruct.Q6_K | 10.36 | 132.80 | 12.82 | 158.62 | 2 |
| gemma-3-4b-it.Q4_K_M | 9.82 | 16.49 | 9.90 | 16.59 | 2 |
| sarashina2.2-3b-instruct-v0.1-Q8_0 | 9.55 | 77.33 | 9.83 | 133.94 | 2 |
| gemma-3-4B-it-QAT-Q4_0 | 9.48 | 16.16 | 9.48 | 16.16 | 1 |
| Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distill-heretic-v3-IQ4_NL | 9.21 | 124.10 | 9.21 | 124.10 | 1 |
1–50 of 75 rows
1 / 2
Head-to-Head Record
1–50 of 345 rows
1 / 7
Performance by App Version
ImprovedRegressed