FDE Times: NVIDIA's garak — one pip install, one scan, then point it at the client's endpoint
A 7 November FDE Times tools guide walks FDEs through NVIDIA's open-source garak LLM vulnerability scanner: install with pip, connect via the REST generator to a client's chatbot, and read PASS/FAIL as per-probe detector rates — not as proof of immunity.
Security questions about an LLM chatbot usually arrive after the demo. FDE Times' 7 November tools guide covers garak (Generative AI Red-teaming and Assessment Kit), the open-source scanner from NVIDIA and the community that tests applications — not just bare models — for failure modes such as data leaks and ignored instructions.
Installation is python -m pip install -U garak. A first local scan against Hugging Face gpt2 shows the output format; connecting to a real client chatbot uses the REST generator and a JSON config that maps the request body and which response field the detectors should score. Databricks' write-up on applying garak stresses the same point: you describe the client's API; garak does not guess it.
PASS and FAIL only report the share of outputs a detector flagged (for example FAIL on 3 of 20). The docs give no pass threshold, several detectors can score the same probe differently, and false positives are common — so open every FAIL before putting it on a slide, and never run probes on production without written scope, environment and timing.
For FDEs, the practical bar is a rerunnable scan with per-probe rates and clear limits: which probes, which environment, and that a PASS is not the same as safe.
Source: fdetimes.net