Documentation

openai.GenericClient$stream

Any OpenAI-compatible chat-completions endpoint: vLLM, LM Studio, TGI, llama.cpp, Together, Fireworks, a corporate gateway.

Reference version

Signature

class openai.GenericClient$stream

Any OpenAI-compatible chat-completions endpoint: vLLM, LM Studio, TGI, llama.cpp, Together, Fireworks, a corporate gateway.

Two things separate it from openai.ChatClient:

  • base_url is REQUIRED. An unset base URL is a configuration error, not a silent fallback to api.openai.com — sending a private prompt to OpenAI because a variable was unset is the worse failure.
  • all-text messages are sent as content: "…" rather than content: [{"type":"text",…}]. Many compatible servers reject the array form outright; media-bearing messages still use parts. It also sends no stream_options (undocumented outside first-party OpenAI) and requires no credential — a local server usually has none.

Source:<builtin>/openai/generic.bamlbytes 0–0

Fields

model

string | null

base_url

ai.Credential$stream | null

api_key

ai.Credential$stream | null

request_timeout_ms

int | null

time_to_first_token_timeout_ms

int | null

request_body

baml.json.json$stream | null

headers

map<string, string> | null

query_params

map<string, string> | null

temperature

float | null

max_tokens

int | null

top_p

float | null

stop

string[] | null

seed

int | null

capture_wire

bool | null