Llama 3.1 8B tops Wazuh, OpenSearch on login anomalies

Meta's Llama 3.1 8B hit 89.3% accuracy detecting authentication anomalies in a new test, beating Wazuh's 52.0% and OpenSearch's 49.3%, per the arXiv paper.

Security teams that lean on Wazuh or OpenSearch for login monitoring are trusting tools that missed most of the true anomalies in this test. Wazuh's false negative rate was 68.6%, OpenSearch's was 74.5%, against 11.8% for Llama.

The researchers built their own testbed: endpoint-specific login logs spanning normal, borderline, and outright anomalous behavior, scored against one shared severity ground truth. Llama's F1 score came in at 91.8%.

The real gap shows up on the hard cases. On borderline anomalous logins, the ones that don't trip an obvious rule, Llama caught 80% of them.

Wazuh caught 20% of those same borderline cases. OpenSearch caught 15%.

Qwen 2.5 7B, the third model tested, wasn't the most accurate of the three. It had the lowest inference latency and produced valid structured output 100% of the time. That matters if the output feeds an alerting system instead of a person's eyes.

Neither Wazuh's rule engine nor OpenSearch's statistical model was built to weigh context the way a language model does, and the borderline numbers show it. The paper doesn't test live production traffic. It also skips the log volumes and attack types Wazuh and OpenSearch were actually tuned for in the field. A follow-up test against real production traffic would settle whether the gap holds outside a controlled testbed.