RSI-Jev

RSI-Jev

System One models, built by AI scientists.

AutoScientists, our AI research agents, propose each change, predict its result before running it, and retire their own champions. Every release and every failed experiment is published.

9releases in 41 days
801experiments, all published, failures included
65.64Decision Index 0.3 public set, v6.1-VL 27B
  1. v1.0Sep 25Probabilities in one pass
  2. v2.0Sep 26A confidence head
  3. v2.1Sep 27Better everywhere it was checked
  4. v3.0Sep 28The first RL win
  5. v4.0-VLOct 1Reads images
  6. v5.0-VLOct 220 of 32 layers
  7. v6.0-VLOct 6Chooses its depth
  8. v6.1-VLOct 7–8Averages two fine-tunes, at 4B and 27B
  9. v7.0-VLNov 5–6Reads the options both ways, at 4B and 3B

How it’s built

The agents run the research.

Each change starts as a hypothesis and ends as a release or a published failure.

  1. HypothesisAn agent proposes a change: new data, a new loss, a new architecture.
  2. PredictionIt writes down the result it expects before any GPU time is spent.
  3. ExperimentIt trains the change and measures it against the current champion.
  4. Keep or retireA winner becomes the next release. Everything else is published with its numbers.

How each release was madeEvery experiment, and why it failed

What the models do

One pass. A probability for every answer.

A System One model reads the question and decides. It writes no text and spends no reasoning tokens, so there is nothing to parse.

A round road sign showing a speed limit of 80.
image + question

Which speed limit number is written on the object in the image?

probabilities61 ms
8099.7%
50<1%
none<1%
2 more answers<1%

A chat model writes instead:{"speed_limit": "80"}11 tokens · 333 ms · no probability

Recorded with v4.0-VL 2B on one GB10. The chat model is Qwen3.5-2B, writing its answer as JSON.

Yes/no, pick one, rateTyped questions about text and up to four images.
CalibratedA probability for every option, fitted so confidence matches accuracy.
Jev-compatiblePOST /v1/systemone. Existing Jev code works once its base URL points here.

v7.0-VL 4B · November 5

A new way to read the options.

v7.0-VL repeats the option list once and lets the question and the options see each other in both directions. It starts from the instruction-tuned Qwen3.5-4B and was trained once, end to end, on data the loop built round by round for its weakest tasks.

60.63Decision Index 0.3v6.1-VL 4B 50.98
0.811held-out setv6.1-VL 4B 0.728
0.502MMLU-Prov6.1-VL 4B 0.462
0.839held-out imagesv6.1-VL 4B 0.834

Decision Index: public index, full run, effort unset (the released files read 60.61). The rest at 32 layers, on the same evaluation sets with the same code. Better calibrated (final ECE 0.039, against 0.048); KoBBQ is lower (0.903, against 0.937). The same request gets the same answer on every run.

The model on Hugging FaceThe 3B cut, at layer 20How it was made

v6.1-VL 27B · October 8

The same recipe at 27B.

Two fine-tunes of Qwen3.8-27B, averaged, with answer heads at layers 48, 56 and 64. It runs on one 96 GB GPU.

65.64Decision Index 0.3v6.1-VL 4B 50.98
0.808held-out setv6.1-VL 4B 0.728
0.660MMLU-Prov6.1-VL 4B 0.462
0.858held-out imagesv6.1-VL 4B 0.834

Decision Index: public set, effort auto; 65.46 with effort unset. The official score is pending. The rest at the deepest layer, on the same evaluation sets. Final ECE 0.064, against 0.048 for the 4B.

The model on Hugging FaceHow it was made

v6.1-VL 4B · October 7

Two fine-tunes, averaged.

v6.1-VL is the average of v6.0-VL and a second fine-tune of the same base. Nothing was trained after the average; the temperatures were refit.

50.98Decision Index 0.3v6.0-VL 46.23
0.728held-out setv6.0-VL 0.695
0.462MMLU-Prov6.0-VL 0.443
0.834held-out imagesv6.0-VL 0.825

Both models on the same evaluation sets with the same code, at 32 layers. Calibration is a little worse: final ECE 0.048, against 0.036.

The model on Hugging FaceHow it was made

On the public Decision Index.

RSI-Jev v6.1-VL, 4B50.98
ezjev 4B s250.82
RSI-Jev v6.0-VL, 4B46.23
jiwo 4B45.76
Nox 4B44.21
JPT-4B42.82

Decision Index 0.3, public index, board of 2026-10-06; 4B models and smaller. RSI-Jev: full runs with the pre-release 0.3 kit.

v6.0-VL 4B · October 6

Fast when it’s easy. Deep when it’s hard.

Choose how many layers it uses: 16, 20 or 32, in one pass. Effort is layers, not tokens.

16, 20 or 32layers
Variesbetween medium and high
Mixed trafficdeeper only when unsure

Answers at layer 16 when it is nearly sure, at 20 when it is sure enough, and at 32 otherwise.

Questions with images always use all 32 layers. Every response reports the layer that answered.

Easy questions stop early. Hard ones go deep.

easy · BANKING77

“I need a refund?”

Which intent?

16 layersrequest refund99%
hard · WinoGrande

“The woman used a gentle technique when painting the urn but applied more pressure when painting the vase, because the _ was more fragile.”

Urn or vase?

16 layersVase51%
32 layersUrn95%

Real outputs of the released model on Decision Index items. BANKING77 is CC BY 4.0; WinoGrande is CC BY.

At release, the best 4B model on the public Decision Index.

RSI-Jev v6.0-VL, 4B46.24
JPT-4B, 4.7B43.04
Jet v6.2, 4.7B42.60
Decider 4B, 4.7B40.70
RSI-Jev v5.0-VL, 3B38.38

v6.0-VL at release: Decision Index 0.2.1, public board of 2026-09-28. Full run, with rows that overlap our training data removed; 14th of 71 overall. Sizes are served parameters.

The release film.

v5.0-VL 3B · October 2

It doesn’t need the whole LLM.

A language model’s deepest layers mostly shape the next word. A System One model never writes, so v5.0-VL keeps only the first 20 of Qwen3.5-4B-Base’s 32 layers.

layers 1–20 runlayers 21–32 are not in the checkpoint
0.760decision suite read at layer 20, against 0.761 at layer 32
0.932says “unknown” when a question has no answer (KoBBQ). v4.0-VL: 0.179
38.38Decision Index, full run: best at 3B and under on the board at the time

v4.0-VL 2B · October 1

It reads images.

Send one to four pictures with a request and ask the same typed questions. On five held-out image benchmarks it answered 80.3% correctly; with the images blanked out, 45.8%.

36 decisions · median 76 ms each

Recorded with v4.0-VL, misses marked ✗. Across all 143 items we ran (89 images), 187 of 227 answers were right.

Hover or tap a picture for the full question and every option. Times are whole calls on one GB10. Images: VisA (Zou et al. 2022) and CLEVR (Johnson et al. 2017), CC BY 4.0, resized; road and object photos from Wikimedia Commons via reinhard-z/vision-jev: stop sign, speed sign and bicycle public domain (Dori, P.Ctnt, Ingolfson), teddy, dog, car, bag and pedestrians CC0, boy with ball CC BY 2.0 (Evgeniy Isaev), red and green lights and leaves CC BY-SA 3.0 (Jacklee; Wald1siedel), cat CC BY-SA 4.0 (Grendelkhan). The crops of the CC BY-SA photos are shared under the same licences. Charts, screens, receipt and shapes are ours (CC0). Breakout from hr98w/jev-visual (MIT). Full credits.

Finds factory defects without seeing one.

No defect examples, one photo at a time. You choose where to draw the line.

RSI-Jev v4.0-VL, 2B86.6
Gemma 4 12B82.9
Jev-Omni (Gemma 4 12B + head)81.1

AUROC on VisA: 50 is a coin flip, 100 is perfect. The same 2,162 photos of 12 kinds of parts for all three. Ours 95% CI 85.1–88.1; the other two are scored from their authors’ published predictions.

50%
405/ 1,200defective parts caught
34/ 962good parts wrongly flagged
defective part (1,200)good part (962)bright dots are flagged

Each dot is one photo’s served probability from the benchmark run behind the AUROC, recorded with v4.0-VL. Photos: VisA, Zou et al. 2022, CC BY 4.0. The full gallery, misses included.

It plays Breakout from pixels.

Every frame it gets one screenshot and says which of five lanes the ball is in. The paddle moves there.

Recorded live with v4.0-VL at real speed: 79 ms per round trip, 200 decisions. Game from hr98w/jev-visual (MIT), unchanged.

Before that

Four text releases in four days.

On the same 15-benchmark suite: 0.622, 0.709, 0.736, then 0.756.

  1. v3.0Sep 28 · 2B

    The first release where reinforcement learning helps. It ranks the right memory first 60% more often than its supervised parent (R@1 0.192 → 0.308).

  2. v2.1Sep 27 · 2B

    Better everywhere it was checked. Ahead of v1.0 on three benchmarks it never trained on.

  3. v2.0Sep 26 · 2B

    Trained on decision benchmarks’ train splits. A confidence head rescales every answer without changing which option wins.

  4. v1.0Sep 25 · 0.8B, 2B

    One forward pass, a probability for every option. The zero-shot reference every later release is compared against.

Get started

Three commands.

Install it, serve v7.0-VL on your own GPU, and ask. Serve v7.0-vl-3b for the 3B, the 4B cut at layer 20; on a 96 GB GPU, v6.1-vl-27b --effort auto for the 27B.

1
pip install "rsi-jev[fast,vision] @ git+https://github.com/Shanghua-Gao/RSI-Jev"
2
rsi-jev serve v7.0-vl-4b --effort auto
3
curl localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": [{"role": "user", "content": "I was charged twice. Please refund."}],
  "questions": {"refund": {"type": "noul",
    "instructions": "Does the user request a refund?"}}}'

Already using Jev? Point your client at http://localhost:8000. For images, add an images list: see the README.