Migrating an OpenAI app to open-weight models

Moving an app from OpenAI to an open-weight model behind an OpenAI-compatible API is mostly a two-value change:

client = OpenAI(
    api_key="$DAWNSTACK_API_KEY",
    base_url="https://api.dawnstackai.com/v1",   # the only line that changes
)

Your SDK, your retry logic, your response parsing and your streaming loop all stay as they are, because the response is the OpenAI shape. That is the easy 90%. This post is the other 10%, because finding it in production is worse than reading it now.

1. Model names are not portable, and should not be aliased

You cannot pass gpt-4o. The catalogue is open-weight models under their own names, so it is llama-3.3-70b, gemma-3-27b, mistral-nemo and so on.

GET /v1/models is public and needs no key, so you can diff it against whatever your code currently sends before changing anything.

What you should not want is a provider that quietly maps gpt-4o onto whatever it considers equivalent. That changes your output and your bill with no way for you to know which model answered. If you are evaluating providers, ask what happens when you send a model name they do not serve. A 404 is the right answer.

2. Three parameters need checking, not assuming

Full passthrough is normal now: tools, response_format, top_p, stop, seed and the penalties are forwarded to the provider. But support is per model, and it is worth knowing which way a provider fails when a model does not support one.

We return a 400 naming the parameter rather than dropping it. Expect that as a new error class in code that appeared to work before, and treat it as a feature: the alternative is a 200 with prose and no tool call, which is indistinguishable from a model choosing not to act.

Of the 46 models we serve, 36 do tool calling and 28 do structured output, measured with real requests rather than copied from a catalogue. Check the model you have in mind before you port the code that depends on it.

3. max_tokens means something different on a reasoning model

This is the one that surprises people.

Models that reason before answering spend the output budget on the reasoning first. Set max_tokens low on one and the budget goes entirely on thinking: you get an empty string and a full bill. We measured one model burning 303 credits at max_tokens: 500 and returning nothing.

Raise the ceiling on reasoning models. It costs nothing, because you are billed on tokens produced rather than on the ceiling you set. And if your replies are short, consider a non-reasoning model instead: 25 of our 46 reason, and for classification, extraction or a one-line answer the non-reasoning ones are usually indistinguishable and dramatically cheaper per request.

4. Recheck cost in your users’ language, not in English

If your product serves an African language, do not carry over a cost estimate built on English traffic. The same meaning is a different number of tokens per language and per tokenizer: Amharic ranges from 1.94x English to 7.69x depending purely on which model you chose.

There is a whole post on choosing a model on that basis. The short version is that the multiplier is a property of the tokenizer, so it does not follow language families and you cannot infer it.

5. What is not supported yet

Stated plainly so you do not discover it after committing: embeddings, vision on most models, audio, and n > 1. Unrecognised request fields are ignored rather than rejected, so a client from a newer OpenAI SDK version still works.

A migration order that works

1. GET /v1/models, pick two candidates on price and capability.
2. Send your ten most typical prompts to each and compare cost per request and output quality. A playground is faster than wiring both into your app.
3. Change the two values in a staging config. Run your existing test suite: if it passes, your parsing and streaming already work.
4. Watch for 400s naming a parameter. Each one is a real capability gap that used to be invisible.
5. Re-measure max_tokens and cost on real traffic before you decide the migration saved you anything.

Step 2 is the one people skip and the one that decides whether this was worth doing.