Verified run· 2 of 3 tasks regraded from the event log; coding tasks not regraded

Qwen3.5-2B Q4_K_M

suite-standard@1.0 · run by Joshua Li · Nov 15, 2026

$ uv tool install "git+https://github.com/holaboss-ai/holaOS@feat/benchclub-cli#subdirectory=packages/benchclub"
$ benchclub login && benchclub run qwen3.5:2b-q4_K_M
Pass rate
44%
4 of 9 checks
Speed
145.4 tok/s
decode, short prompt
At 8K context
136.8 tok/s
decode
First token
42.7 ms
short prompt

Tasks

Each task runs 3 times and is graded by a program, not by another model.

  • Codingcode-intervals

    Writes a Python function. Unit tests decide the pass.

    1/3Not regraded
  • Tool usetool-flight

    Calls a flight-search tool. The arguments have to be right.

    1/3Regraded from log
  • Long contextneedle-8k

    Finds one fact hidden in about 8K tokens of text.

    2/3Regraded from log

A community run on the creator’s own machine. It counts toward their profile, not toward headline numbers.

PassedFailedNot supported

Against BenchClub cloud

This model has no cloud result yet, so there’s nothing to compare against.