Preview. These maps are our reading of public sources and haven't been checked with the projects' maintainers yet.
jevals (Openlayer)
Agent evals and guardrails expressed as Jev decisions (one request per trace), with backends for Jev via TypeSafe or Vercel, OpenRouter decision models, local Kev or Laya, Eikos on your GPU, or an emulating chat LLM.
Repo pushed 2026-10-15; PyPI jevals 0.1.4 (all versions uploaded 2026-09-20). Show HN 2026-09-20 with 47 points. (checked 2026-10-25)
In The Jev ecosystem
Where it stands apart
- Which models answer its questions? Pick the first that applies. Jev or other models, your choice (14% of 21 others)
- Where do the decisions run by default? no default: you choose a local or hosted backend (14% of 21 others)
- Can you use it without a TypeSafe account? yes, through a gateway or another provider (24% of 21 others)
What it shares with most of the community
- Is it actively maintained? active (62% of 21 others)
- How is the main thing licensed? permissive open source (MIT/Apache/BSD) (86% of 21 others)
- Can it run entirely on your own device or infrastructure, without the maker's servers? yes (52% of 21 others)
- Do you need an account to use it? no account (86% of 21 others)
- Who is it for? developers (95% of 21 others)
- Which shared standard or format does it mainly build on? TypeSafe's System One API (state + typed questions, /v1/systemone) (81% of 21 others)
- Which question types does it handle? Jev's three: noul, choice, score (67% of 21 others)
- Does it train on Jev's outputs? (TypeSafe's customer agreement forbids using output to distill or build competing models.) it doesn't train models (68% of 19 others)
- Are its answers typed values rather than generated text? (identity question) yes, typed answers only (67% of 21 others)
Nearest neighbors
- jevper 75% of 16 shared answers match
- decide 69% of 16 shared answers match
- onesie 69% of 16 shared answers match
Every answer
| Question | Answer | Source |
|---|---|---|
| Is it actively maintained? | active | source 2026-10-25 |
| How is the main thing licensed? | permissive open source (MIT/Apache/BSD) MIT | source 2026-10-25 |
| How is it mainly sustained? | company with a paid product, tier, or ads Published by Openlayer, an eval/observability company. | source 2026-10-25 |
| Can it run entirely on your own device or infrastructure, without the maker's servers? | yes With the Laya in-process or Kev backend. | source 2026-10-25 |
| Do you need an account to use it? | no account pip library; needs a provider key (Vercel, TypeSafe, OpenRouter) or local backend, no Openlayer account. | source 2026-10-26 |
| Who is it for? | developers | source 2026-10-25 |
| Which shared standard or format does it mainly build on? | TypeSafe's System One API (state + typed questions, /v1/systemone) Jev decisions with several backends. | source 2026-10-26 |
| Which models answer its questions? Pick the first that applies. | stands apart Jev or other models, your choice | source 2026-10-25 |
| What is the main thing it ships? | a library, CLI, or SDK for calling models | source 2026-10-25 |
| Which question types does it handle? | Jev's three: noul, choice, score | source 2026-10-25 |
| Does it check whether its probabilities can be trusted? Pick the first that applies. | no, it passes probabilities through without measuring them Notes that the emulated LLM backend returns 0/1 instead of calibrated values. | source 2026-10-25 |
| Where do the decisions run by default? | stands apart no default: you choose a local or hosted backend | source 2026-10-25 |
| Does it train on Jev's outputs? (TypeSafe's customer agreement forbids using output to distill or build competing models.) | it doesn't train models | source 2026-10-25 |
| Can you use it without a TypeSafe account? | stands apart yes, through a gateway or another provider | source 2026-10-25 |
| Are its answers typed values rather than generated text? (identity question) | yes, typed answers only Except the LLM-emulation fallback. | source 2026-10-25 |
| Does it publish head-to-head numbers against Jev? | yes, head-to-head numbers against Jev Compares backends (Jev, Eikos) and LLM judges on cost, time and agreement. | source 2026-10-25 |
Sources
This page was derived from:
- GitHub · github.com/openlayer-ai/jevals · read 2026-10-25
RSS for this project · Its own release feed
Are you a maintainer? Claiming and correcting projects is coming soon.