Using system-one models inside high-throughput data pipelines

Jev (TypeSafe's system-one model) is a quantum leap forward for data processing pipelines, if harnessed correctly. We used Jev to accelerate an entity resolution pipeline to join donors and committees across Ohio campaign finance data. Hank-assisted resolution pipelines that use Jev as the workhorse reduced model costs by 99.56% and increased measured call-time throughput to 7.35× the baseline while retaining near-Fable quality.

But it's not all roses. Applying the wrong signal or prompting the wrong way causes large mistakes at scale. No free lunches, only subsidised ones.

We'll talk about the results, explain the task of entity resolution, and at the end we'll provide some prompts and principles that can work for integrating models like Jev into AI data pipelines.

Results

On a segment of the Ohio dataset that we selected to be difficult, using Fable to join entities costs $9.83 for 1.2 MB of prepared evidence, and takes 844.64 seconds.

An optimized hank (like the one below) is an alternative to direct comparisons using a model or a code-based approach. Here, it reduces model costs to $0.3012 for 715.08 seconds (a 32.6× reduction in cost but about the same time).

Using only Jev to adjudicate drops accuracy significantly, but Jev as the primary workhorse inside a five-family review stage with Luna provides 226x lower cost ($0.0000361 per KB), 7.35x higher throughput (10.5 KBps in call-seconds against Fable's 1.4), while being within 0.5 percentage points of Fable in accuracy.

Jev is the final missing piece for something we set out to do 18 months ago; to scale these workflows up to Terabytes of data.

The job

The task is simple. Can we resolve entities across the Ohio Campaign Finance dataset? Entities here could be donors, committees, companies, etc. This is the task we have been working on as part of a much larger ingestion and data pipelining test designed to measure 4a, our internal superagent that can autonomously build and self-optimize AI data pipelines.

In practice, ER is incredibly difficult. Humans make honest mistakes - middle names are a common example.

We also move houses and jobs, change names, get married, omit information, and overall shenaniganize our lives.

Then there's (alleged) hostile obfuscation - large donors and companies attempting to spread out detection with different entity names, honest looking mistakes, etc etc.

Our ER pipeline resolves these problems using atoms - bundles of equivalent donor evidence. We put rows together into atoms, and models decide on what these bundles are (a company, a person, a committee, a joint donor or an unnamed record), and how they're related (same person, same household?). As we join more and more atoms, the dataset slowly resolves itself.

Here's one case: 11 rows, five atoms, two people. Ordinary code makes the atoms and keeps every original row. Jev judges which atoms represent the same donor, then whether the two donors share a household. The household is a connection between people, not permission to merge them.

This is where Jev comes in.

If you were so inclined (either rich or a frontier lab), you could have a frontier LLM adjudicate every atom and bundle. However - even if you could pay for all of this (our estimate for the four-file Ohio snapshot is about $9,614 for Fable) - speed makes it difficult to do so. Even the best of us only have about 3 billion seconds of life.

Additionally, we have found that at scale, using cheaper models with additional review steps can surpass the accuracy of even frontier models deployed raw.

Prompts

Here are the things we learned making Jev useful for this task.

1. Telling Jev what counts as enough evidence. Our first questions spent too much time telling it what wasn't proof.

This was the original yes criterion for a household link:

This is the criterion we replaced it with:

The revised instructions also say that DR and DRIVE don't break an address match, and that a shared employer, business address or PO box doesn't establish a household. On the same 13 development questions, Jev went from answering no to all 13 to yes to all 13, matching all 13 reference answers. Those are development results, not the final benchmark score.

2. We made the same change to identity. Here is the actual revised same criterion:

Much like prompting in the 3.5-turbo days, more examples and more specific choices make things easier. Same for humans: we all prefer MCQs over essays!

Conclusion

More work remains to be done in integrating intelligence classifiers across data problems, especially when it comes to pipelining the raw performance on offer. This is the largest change we've seen in architecture since the flash (gemini, deepseek, glm) series of models to change how we structure data work in production. We are already seeing significant bumps in accuracy, cost and speed across other tasks like mining, selection, sanitization, and a lot more. We'll share results as we can make them ready for publication.

Data

You can download the publication-safe experiment data and the Ohio campaign-finance sources here.

• Costs, clocks and workload data
• Research results, including repeat runs and other trials
• The illustrated case, with invented identifying values
• Exact prompt excerpts
• Runnable seven-row example and full offline report — extract the bundle, then run bun example.ts --dry-run from the publication folder. This makes no model call. The small example demonstrates classification and joining labels back to rows, not the full resolver.
• Ohio's public campaign-finance search and downloads

Want to try this on your own data? Start with Jev's typed questions, and use Hankweave for the reasoning steps.