← Quantizations
Quant · GGUF · 5 variants

securityllm-gguf

Quantization of ZySec-AI/SecurityLLM .

HF Orionfold/SecurityLLM-GGUF License free apache-2.0 Published

What this model does

ZySec-AI's SecurityLLM is a 7B security-domain chat model (Zephyr format). This release ships five GGUF variants (Q4_K_M at 4.1 GB and 47.7 tok/s up to F16) so it runs offline on consumer hardware, each carrying a four-axis Spark-measured card: wikitext-2 perplexity, sustained tok/s, thermal-envelope minutes, and a CyberMetric score. Unusually, the smallest quant (Q4_K_M) tops the bench — Orionfold's contribution is the distribution + measurement layer that surfaces that; ZySec-AI did the security fine-tune.

Use cases

  • Offline security-domain chat and concept Q&A on consumer hardware
  • A study aid for security certifications and terminology
  • Picking a quant variant by workload shape, not just RAM budget

Audience — Local-LLM power users and security learners who want an offline cybersecurity chat model on a consumer GPU — for study and exploration, not operational security decisions.

Spec matrix

Ranks within each column drive the heatmap. Lower perplexity, higher throughput, higher vertical eval — the sweet-spot row balances all three.

Vertical bench: CyberMetric (n=50, mcq_letter)
Variant Perplexity ↓ Spark tok/s ↑ Vertical eval ↑
Q4_K_M Sweet spot 7.3998 47.66 0.40
Q5_K_M 7.3142 39.95 0.38
Q6_K 7.3132 34.96 0.36
Q8_0 7.3068 30.34 0.36
F16 7.3009 17.45 0.34

Methods

Read the field note Orionfold/SecurityLLM-GGUF on Spark — five cyber variants, CyberMetric mini-eval, MCQ letter scoring Five GGUF variants of ZySec-AI/SecurityLLM measured on a DGX Spark — Q4_K_M scores 40% on CyberMetric MCQ at 47.7 tok/s and 4.1 GB; the smaller variants matched or beat F16's 34%. Third vertical card; zero fieldkit source changes. Open article

Known drift

Disclosed limitations with explicit bounds — the scope is named, not implied.

CyberMetric accuracy is modest (4-choice MCQ, n=50)
CyberMetric (n=50, mcq_letter) lands 34–40% — above the 25% random baseline for 4-choice MCQ but modest, and the 50-question sample makes the variant ordering statistically loose. A 7B ceiling, not a quant failure.
Not a security tool or advisory source
A 7B chat model inherited from the upstream base — for study and concept Q&A, not vulnerability assessment, incident response, or operational decisions. No security-grade validation is claimed.