quant · cyber
securityllm-gguf
| Variant | Perplexity | tok/s | CyberMetric (n=50, mcq_letter) |
|---|---|---|---|
| Q4_K_M sweet spot | 7.400 | 47.7 | 0.40 |
| Q5_K_M | 7.314 | 40.0 | 0.38 |
| Q6_K | 7.313 | 35.0 | 0.36 |
| Q8_0 | 7.307 | 30.3 | 0.36 |
| F16 | 7.301 | 17.4 | 0.34 |
Perplexity lower = better; tok/s measured on the DGX Spark (GB10, 128 GB unified).