Fine-tuned models are outperforming frontier APIs on cost, speed, and quality simultaneously
The assumption that better AI requires paying more for frontier APIs is breaking down in production. Practitioners across clinical documentation, compliance, and customer service are reporting that custom fine-tuned and distilled models now beat frontier models on all three dimensions at once.
Jesse Zhang, who leads Decagon, put the outcome plainly: after fine-tuning open-source models for customer service, his company now runs 90 percent of its workflow on open-source, and the result is better performance, lower cost, and lower latency at once. That three-way improvement is the finding that makes this moment unusual. Historically, gains on one dimension came at expense on another. That tradeoff is no longer holding.
Kyle Corbitt, whose work sits at the fine-tuning infrastructure layer, puts numbers on the pattern. Custom fine-tuned models can bring latency down to about 30 percent of what frontier models deliver, with similar or higher quality. On cost per token, the improvement runs to at least an order of magnitude, and often more. Those figures align with what practitioners in different sectors are reporting from their own production systems.
Shiv Rao, whose company Abridge operates in clinical documentation, says 40 percent of its model outputs now come from in-house models built through distillation and fine-tuning. He expects that share to reach 60 percent within a month, as a newly distilled open-source model replaces a frontier model in production. Brian Armstrong, Coinbase’s chief executive, makes the same point from a compliance angle: a fine-tuned open-weight model trained on 100,000 proprietary Coinbase compliance cases can outperform a frontier model on that specialized task. The pattern repeats across domains.
We can typically get you know, latency down to about 30% of what you get from using a frontier model with again similar or usually higher quality than what you were getting from the frontier model. Kyle Corbitt
The price gap between frontier APIs and the alternatives has become hard to ignore. Chamath Palihapitiya has noted the contrast between $56 per million input tokens at the high end of the frontier market and roughly 50 cents for competing models. Satya Nadella, Microsoft’s chief executive, described a comparison between approximately $50 per million output tokens for OpenAI against an estimate that DeepSeek can go as low as 15 cents per million tokens of output, a figure he rounded to 60 cents, calling the difference a 99 percent cost reduction. Marc Andreessen adds that distilling a model costs approximately 2 percent of the original pre-training cost, which means the barrier to producing a capable custom model is far lower than the barrier to training from scratch.
The speed dimension has its own mechanics. Simon Mo points out that proprietary model APIs typically offer only two speed settings, regular and fast, while open-weight models deployed through providers can offer up to 10 different speed tiers, reaching 400 tokens per second and up to roughly 500 in many cases. That ceiling is two to three times faster than the fast mode available from proprietary APIs today. For latency-sensitive applications, that range of control matters in ways that a binary switch cannot provide. Tulsee Doshi, who works on search at scale, confirms the user-side reality: even when a model is significantly better in quality, users do not tolerate the wait, and latency degrades outcomes in live experiments regardless of quality gains.
Practitioners are also routing by task rather than defaulting to a single frontier model for everything. Swyx, an AI researcher, describes using a self-hosted 800-million-parameter model for query classification rather than routing that call to a frontier API, noting that the smaller model does not need to be jammed into a large prompt or subjected to a one-second latency API call. For simple classification tasks, a fine-tuned model at roughly one billion parameters can recover about 95 percent of frontier performance. Ahmad Awais describes a common workflow in which teams use a high-quality frontier model once to generate a baseline, then rely on cheaper models for all subsequent development. The frontier model becomes a calibration tool rather than a production dependency.
Jensen Huang has observed that 80 percent of AI-native companies already use open models. Simon Mo traces the shift to a threshold crossed about a year before his remarks, when smaller application companies concluded that building competitive AI products required their own mid-training and post-training work. Closed-source vendors do not permit that access. Open-weight models do. Brendan Foody draws the trajectory to its endpoint, predicting that the majority of inference in five years will run on open-source, custom fine-tuned, or distilled models rather than frontier APIs. The cost curve and the capability curve are moving in the same direction at the same time, and the practitioners building on top of both are already adjusting their infrastructure accordingly.