solo_jev

0814wdwd/solo_jev is ranked 16,801 among these 8 peers; typesense/typesense-js is ranked 16,799.

It arranges **rows and fields** so a prefix-caching backend can reuse more input computation, then returns decisions in your original row order. The backend receives the **complete record on every request**. **Measured on one RTX 4090:** up to **11. 7× throughput over original order** with long shared context, and **3. 28× over NDV sorting** with correlated fields. Both are reproducible synthetic workloads; see the full conditions below. Use it for structured workloads with repeated policies, account context, catalog attributes

, or correlated fields. The included backend serves **AutoTrust JEV-9B**, an open decision model built on Qwen3. It returns binary, categorical or scored decisions with probabilities. The layout optimizer also works independently with your own backend. Quickstart Install from this source checkout (Python 3. For inference, connect to a running JEV-9B service or deploy one on your GPU. With the default local service: The client downloads only the pinned model's two small decision-head metadata files on first inference

. Model weights, vLLM and Torch stay on the server. **Try the optimizer with no GPU or network:** **JSON is an input, too:** JSON text, JSON/JSONL files, nested objects, DataFrames, and NumPy arrays share the same engine. See the input contract for schema and type rules. Why can a one-token decision still be slow? A decision model may return only one label, but it must first process the full contract, policy, user context or candidate. On JEV-9B, median

request latency rose from 0. 280 seconds at roughly 1K input tokens to 0. 5K, even though every request produced exactly one token. SOLO keeps every field and leaves the model unchanged. It reorganizes repeated content inside an already-ready batch so the inference server can reuse more of the input. Across all 1,088 decisions from 64 selected ContractNLI contracts, original-order accuracy was 66. Completing the same 17 batches of 64 took 293. 64 seconds with the original

layout and 115. The server timings show where that reduction happens. The median interval from scheduling to the first decision token fell from **726. Every request emits one decision token, so the subsequent decode interval was **0. 0 ms** in all 2,176 original/SOLO request measurements. The speedup disappears when APC is disabled. 19% of SOLO prompt tokens had an earlier exact prefix, 73. 89% remained reusable after 528-token block quantization, and vLLM reported 72. Full protocol, cache

controls and statistics · Timing aggregate · Full-coverage aggregate Two reproducible demos These are **controlled synthetic workloads measured on a real RTX 4090** with JEV-9B BF16, vLLM 0. 0, prefix caching enabled and concurrency 4. The model, complete input records, and concurrency are held constant across layouts. Each method starts in a fresh cache namespace; reported time includes planning, serialization, HTTP and the first cache fills. Repeated context: move the unique ID out of the way Many records share policies

or account context, but an ID-first serialization breaks the reusable prefix. This demo deliberately varies the length of repeated