Quant · GGUF · 5 variants
securityllm-gguf
Quantization of ZySec-AI/SecurityLLM .
Spec matrix
Ranks within each column drive the heatmap. Lower perplexity, higher throughput, higher vertical eval — the sweet-spot row balances all three.
| Variant | Perplexity ↓ | Spark tok/s ↑ | Vertical eval ↑ |
|---|---|---|---|
| Q4_K_M Sweet spot | 7.3998 | 47.66 | 0.40 |
| Q5_K_M | 7.3142 | 39.95 | 0.38 |
| Q6_K | 7.3132 | 34.96 | 0.36 |
| Q8_0 | 7.3068 | 30.34 | 0.36 |
| F16 | 7.3009 | 17.45 | 0.34 |