AI One Lab / Study 2
Can an open-weight decision model match frontier performance for an enterprise?
A measured study of task-specific fine-tuning for an open-weight decision model on enterprise workflow classification.
Overview
Paper 1 asked whether an open-weight model could handle an enterprise decision task. This follow-up tests how much that model improves when it is trained for the task. AI One Labs trained Clef on 4,998 synthetic examples and evaluated it against matched stock Clef and earlier hosted-model results.
The result is a task-specific classifier, not a claim that a small model has become a general-purpose frontier model.
What the paper covers
A clear gain over the matched stock model
Fine-tuned Clef classified 1,906 of 1,974 historical benchmark requests correctly (96.56%), up from 1,827 (92.55%) for matched stock Clef. It corrected 86 mistakes while introducing seven.
Observed performance in the frontier range
The earlier GPT-6 Luna high-reasoning result was 96.10% on the same cohort. The small observed difference does not establish superiority or formal equivalence.
A separate sealed test supports the direction
On a 999-example test from the training recipe, accuracy rose from 98.50% to 99.90%. The paper reports $156.93 in primary data-production cost and 4.07 hours of training on one RTX 4090, with engineering and serving costs additional.
How to read the findings
This is evidence for a defined 50-category classification task. The historical benchmark had been used during development; the sealed test comes from the synthetic training recipe. The paper reports both, along with methods, error analysis and cost boundaries.