Quant · GGUF · 5 variants
saul-7b-instruct-v1-gguf
Quantization of Equall/Saul-7B-Instruct-v1 .
Spec matrix
Ranks within each column drive the heatmap. Lower perplexity, higher throughput, higher vertical eval — the sweet-spot row balances all three.
| Variant | Perplexity ↓ | Spark tok/s ↑ | Vertical eval ↑ |
|---|---|---|---|
| Q4_K_M | 5.9864 | 29.43 | 0.62 |
| Q5_K_M Sweet spot | 5.9380 | 20.19 | 0.72 |
| Q6_K | 5.9250 | 22.39 | 0.68 |
| Q8_0 | 5.9138 | 7.30 | 0.66 |
| F16 | 5.9165 | 10.88 | 0.68 |