Preview. These maps are our reading of public sources and haven't been checked with the projects' maintainers yet.
JevBench (Benchmark Heaven)
Independent benchmark and leaderboard for Jev-class decision models: accuracy, calibration, latency and cost, with Jev 1.13/1.14 as references and separate open-weight and hosted-API boards.
Board shows 'method v1.7, updated 2026-10-25', 223 ranked open-weight systems; repo fstandhartinger/jevbench (MIT) pushed 2026-10-15, published release v1.6.1 on 2026-10-06. (checked 2026-10-25)
In The Jev ecosystem
Where it stands apart
- Can it run entirely on your own device or infrastructure, without the maker's servers? partly (some features need the maker's servers) (19% of 21 others)
- Which models answer its questions? Pick the first that applies. none: it measures, compares, or catalogs Jev-style systems (10% of 21 others)
- What is the main thing it ships? a benchmark or directory (10% of 21 others)
- Where do the decisions run by default? not applicable (benchmark or directory) (10% of 21 others)
- Are its answers typed values rather than generated text? (identity question) not applicable (benchmark or directory) (10% of 21 others)
What it shares with most of the community
- Is it actively maintained? active (62% of 21 others)
- How is the main thing licensed? permissive open source (MIT/Apache/BSD) (86% of 21 others)
- How is it mainly sustained? individual side project, no revenue (55% of 20 others)
- Do you need an account to use it? no account (86% of 21 others)
- Who is it for? developers (95% of 21 others)
- Which shared standard or format does it mainly build on? TypeSafe's System One API (state + typed questions, /v1/systemone) (81% of 21 others)
- Which question types does it handle? Jev's three: noul, choice, score (67% of 21 others)
- Does it train on Jev's outputs? (TypeSafe's customer agreement forbids using output to distill or build competing models.) it doesn't train models (68% of 19 others)
- Can you use it without a TypeSafe account? yes, no TypeSafe account involved (62% of 21 others)
Nearest neighbors
- DoubtBench 88% of 16 shared answers match
- Kev 63% of 16 shared answers match
- snap 63% of 16 shared answers match
Every answer
| Question | Answer | Source |
|---|---|---|
| Is it actively maintained? | active | source 2026-10-25 |
| How is the main thing licensed? | permissive open source (MIT/Apache/BSD) Code MIT; the board's sealed items are not public. | source 2026-10-25 |
| How is it mainly sustained? | individual side project, no revenue Benchmark Heaven describes itself as an open source hobby project. | source 2026-10-26 |
| Can it run entirely on your own device or infrastructure, without the maker's servers? | stands apart partly (some features need the maker's servers) Harness and public items run locally; official scores use sealed inputs that can't be fully rerun. | source 2026-10-25 |
| Do you need an account to use it? | no account Public leaderboard and MIT harness; no sign-in needed. | source 2026-10-26 |
| Who is it for? | developers | source 2026-10-25 |
| Which shared standard or format does it mainly build on? | TypeSafe's System One API (state + typed questions, /v1/systemone) Entrants run as Jev-compatible systems via a typesafe adapter. | source 2026-10-26 |
| Which models answer its questions? Pick the first that applies. | stands apart none: it measures, compares, or catalogs Jev-style systems | source 2026-10-25 |
| What is the main thing it ships? | stands apart a benchmark or directory | source 2026-10-25 |
| Which question types does it handle? | Jev's three: noul, choice, score | source 2026-10-25 |
| Does it check whether its probabilities can be trusted? Pick the first that applies. | yes, it publishes calibration metrics (ECE, Brier, or similar) Calibration is half of the headline Capability Score (ECE plus fidelity to gold distributions). | source 2026-10-25 |
| Where do the decisions run by default? | stands apart not applicable (benchmark or directory) | source 2026-10-25 |
| Does it train on Jev's outputs? (TypeSafe's customer agreement forbids using output to distill or build competing models.) | it doesn't train models Requires entrants to disclose training on the public split. | source 2026-10-25 |
| Can you use it without a TypeSafe account? | yes, no TypeSafe account involved | source 2026-10-25 |
| Are its answers typed values rather than generated text? (identity question) | stands apart not applicable (benchmark or directory) | source 2026-10-25 |
| Does it publish head-to-head numbers against Jev? | yes, head-to-head numbers against Jev | source 2026-10-25 |
Sources
This page was derived from:
- GitHub · github.com/fstandhartinger/jevbench · read 2026-10-25
- Web page · benchmarkheaven.com/jev-models · read 2026-10-25
Are you a maintainer? Claiming and correcting projects is coming soon.