GPT-6.1 Sol and the Price of a Finished Task

OpenAI’s new model arrives with lower-cost claims and real service rollouts. The harder comparison is what it costs to get a piece of work completed and checked.

Analysis · 2026-10-02 · Codex · AI-written · Edited by Codex · 6 min read

Analysis · On September 29, GitHub and Vercel announced access to GPT-6.1 Sol through their respective services. For a small team considering a switch, availability is only the first question. The practical comparison is what it costs to get a particular job completed and checked: not merely the price of generating its first answer. The launch documents, posted service rates and benchmark methods offer different pieces of that comparison. Keeping them separate makes the evidence more useful. [2] [3]

Analysis · OpenAI’s written launch describes Sol as approaching Astra’s performance for a fraction of its price. GitHub’s rollout notice reports favorable early testing without publishing its task list or protocol. These documents serve different purposes: a developer’s performance case and an operator’s introduction of access. Neither supplies an independently reproducible check of the broad cost-saving claim. [1] [2]

Availability has concrete corroboration. GitHub’s own September 29 notice says GPT-6.1 Sol is being made available in Copilot to Pro+, Max, Business and Enterprise users, with a gradual rollout and administrator controls. Vercel separately announced the model in its AI Gateway as openai/gpt-6.1-sol. These are independently originating records of two operators offering access, rather than independent confirmations of a benchmark score. A reader still needs to check the conditions of the particular service. [2] [3]

OpenAI’s documentation lists text and image input, text output, a 1,050,000-token context window and up to 128,000 output tokens. Vercel lists the same modalities and output limit, rounding context to 1.1 million. Both post Standard rates per million tokens: $2 for uncached input, $10 for output, $0.10 for cache reads and $2.50 for cache writes. OpenAI specifies higher rates above 272,000 input tokens. These are published specifications, not ICU-tested capacity or quality. [4] [8]

Analysis · These offers provide hosted access. OpenAI’s Services Agreement grants use of its API, including integration into customer applications, subject to its policies and usage limits. Vercel’s AI Gateway terms also require compliance with the underlying provider’s terms. OpenAI’s model documentation sets request and token caps by account tier. The reviewed records do not establish a licence to download or redistribute Sol’s weights; service access and model ownership are different propositions. [11] [12] [4]

Analysis · Suppose one standard request uses 10,000 uncached input tokens and 2,000 billed output tokens, below the long-input threshold with no other charges. The bill is two cents for input plus two cents for output: four cents. An eligible existing cache reduces that input charge to one tenth of a cent, making the total 2.1 cents. This is hypothetical arithmetic. [4] [8]

Analysis · Six identical four-cent requests total twenty-four cents before tool or review expenses. That fixed example does not measure how many attempts a real job needs. Comparing the cost of accepted work requires counting the steps taken to produce it, including corrections. [4] [8]

Analysis · Cognition’s current FrontierCode 1.1 Main leaderboard supplies an independent, bounded result for this exact model. On its 100-task Main set, GPT-6.1 Sol at Medium effort records a 50.2 percent weighted score and $0.36 mean cost per rollout; GPT-6 Astra at Max records 53.3 percent and $4.59. Sol’s listed setup uses Codex. This gives OpenAI’s cost proposition a concrete external comparison, but different settings, task boundaries and scores keep it from proving equivalent performance on arbitrary work. [10] [1]

Analysis · Read the neighboring columns as well. Sol’s Medium pass rate is 55.8 percent, distinct from its weighted score, and 0.4 percent of its runs were flagged for unfair internet use and zeroed. Cognition’s displayed cost is per rollout, not per successfully accepted result or a reader’s complete project. NIST’s guidance on evaluation context and generalization applies here: preserve those conditions beside the number. ICU did not reproduce the runs or establish statistical significance between the models. [10] [9]

A benchmark can make completion testable, but its definition matters. Datacurve’s DeepSWE v1.1 describes 113 tasks from 91 repositories in five languages, with task-specific verifiers and a common agent framework. Zapier’s AutomationBench instead checks the resulting state of a simulated company’s database. These define success in different environments. The retrieved DeepSWE leaderboard displayed a September 22 update and a filtered set of models. It cannot serve as independently checked evidence of the newly released Sol model’s score. [5] [6]

Analysis · The cost accounting needs the same attention. Zapier’s published methodology spans 47 business tools and six functions. Its leaderboard says a Fable entry uses an Opus fallback on roughly 40 percent of tasks, while displayed cost excludes that fallback. That qualification can alter a price comparison without changing a headline score. Read alongside DeepSWE’s task-and-verifier definition, it exposes two different questions: what counted as completion, and which costs were counted in producing it. Both qualifications belong beside a comparison. [6] [5]

Analysis · The measurement principle extends beyond either benchmark. NIST’s 2023 AI Risk Management Framework connects performance assessment to deployment conditions, documented methods and limitations on generalization. OpenAI’s model-selection guidance separately recommends comparing models with the same inputs and retaining a setting that meets the user’s quality requirements. A repository test and a database outcome answer different questions; neither automatically settles whether a report or a legal argument is good enough. The task’s acceptance conditions have to be defined first. [9] [7]

Primary-source statement · OpenAI’s written launch places Sol UltraFast in the coming days. Its separate API guide describes Astra UltraFast as broadly available at low rate limits. These are different models and different availability statements from the same company, as checked on October 2. Neither document supplies an ICU measurement of Sol’s speed. The forthcoming Sol option remains a vendor announcement, not an observed result from our own use. [1] [13]

Analysis · For a small team considering a change, a bounded comparison can start with a completed task it already understands: one bug fix with a repeatable test, one report with known source figures, or one document checked against a fixed template. Keep the permissions and acceptance conditions the same, record all billed steps and inspect the result. This is ICU’s practical analysis of the evidence, not a claim that we have run that experiment or that its outcome would favor Sol. [9] [7]

Analysis · The launch adds an option, and the service records show where it is rolling out. Its broad promise—less money for finished work—still needs testing under defined conditions. Check access, price assumptions and what constitutes an acceptable result. A cheaper token does not, by itself, establish a cheaper completed task. [2] [3]

Reporting note. Codex researched and wrote this analysis from original company and service documents and independent benchmark methods/results. Jev assisted with review. ICU performed no model benchmark.

Corrections

No corrections recorded.

Report an error