Guides

SDK and framework compatibility

Working configuration for the OpenAI SDKs, LangChain, LlamaIndex, the Vercel AI SDK and LiteLLM, plus the two settings that trip people up.

Anything that speaks the OpenAI chat completions API works. Point it at https://api.dawnstackai.com/v1 with a Dawnstack key.

OpenAI SDKs

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DAWNSTACK_API_KEY"],
    base_url="https://api.dawnstackai.com/v1",
    max_retries=3,
)
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.DAWNSTACK_API_KEY,
  baseURL: "https://api.dawnstackai.com/v1",
  maxRetries: 3,
});

The SDKs retry 429 and 5xx with backoff on their own. They do not retry 402, which is correct: an empty balance does not fix itself.

LangChain

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="llama-3.3-70b",
    api_key=os.environ["DAWNSTACK_API_KEY"],
    base_url="https://api.dawnstackai.com/v1",
)
import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "llama-3.3-70b",
  apiKey: process.env.DAWNSTACK_API_KEY,
  configuration: { baseURL: "https://api.dawnstackai.com/v1" },
});

Use langchain-openai, not a provider-specific package. Tool calling and structured output work through it, on models that declare support.

LlamaIndex

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="llama-3.3-70b",
    api_key=os.environ["DAWNSTACK_API_KEY"],
    api_base="https://api.dawnstackai.com/v1",
    is_chat_model=True,
    is_function_calling_model=True,
)

Vercel AI SDK

import { createOpenAI } from "@ai-sdk/openai";
import { streamText } from "ai";

const dawnstack = createOpenAI({
  apiKey: process.env.DAWNSTACK_API_KEY,
  baseURL: "https://api.dawnstackai.com/v1",
  compatibility: "compatible",
});

const result = streamText({
  model: dawnstack("llama-3.3-70b"),
  prompt: "Explain server-sent events in two sentences.",
});

LiteLLM

from litellm import completion

response = completion(
    model="openai/llama-3.3-70b",
    messages=[{"role": "user", "content": "Hello"}],
    api_key=os.environ["DAWNSTACK_API_KEY"],
    api_base="https://api.dawnstackai.com/v1",
)

The openai/ prefix routes it through LiteLLM’s OpenAI handler. Without it, LiteLLM tries to infer the provider from the model name and fails.

Anything else

If a tool takes a base URL and a key, it works. Two settings cause almost every failure:

  • A trailing path. The base URL is https://api.dawnstackai.com/v1, and most clients append /chat/completions themselves. Including it yourself produces a 404 on /v1/chat/completions/chat/completions.
  • A legacy completions call. Some wrappers default to /v1/completions, which is not implemented. Look for a “chat model” flag.

If something still does not work, tell us what client and what version. That is the kind of report that turns into a fix rather than a workaround.