Documentation

openai.GenericClient

Any OpenAI-compatible chat-completions endpoint: vLLM, LM Studio, TGI, llama.cpp, Together, Fireworks, a corporate gateway.

Reference version

Signature

class openai.GenericClient

Any OpenAI-compatible chat-completions endpoint: vLLM, LM Studio, TGI, llama.cpp, Together, Fireworks, a corporate gateway.

Two things separate it from openai.ChatClient:

  • base_url is REQUIRED. An unset base URL is a configuration error, not a silent fallback to api.openai.com — sending a private prompt to OpenAI because a variable was unset is the worse failure.
  • all-text messages are sent as content: "…" rather than content: [{"type":"text",…}]. Many compatible servers reject the array form outright; media-bearing messages still use parts. It also sends no stream_options (undocumented outside first-party OpenAI) and requires no credential — a local server usually has none.

Source:<builtin>/openai/generic.bamlbytes 772–6992

Fields

model

string

base_url

ai.Credential | null

api_key

ai.Credential | null

request_timeout_ms

int | null

time_to_first_token_timeout_ms

int | null

request_body

baml.json.json | null

headers

map<string, string> | null

query_params

map<string, string> | null

temperature

float | null

max_tokens

int | null

top_p

float | null

stop

string[] | null

seed

int | null

capture_wire

bool

Static methods

function

new

(
model: string,
base_url: ai.Credential | null = …,
api_key: ai.Credential | null = …,
request_timeout_ms: int | null = …,
time_to_first_token_timeout_ms: int | null = …,
request_body: baml.json.json | null = …,
headers: map<string, string> | null = …,
query_params: map<string, string> | null = …,
temperature: float | null = …,
max_tokens: int | null = …,
top_p: float | null = …,
stop: string[] | null = …,
seed: int | null = …,
capture_wire: bool = …
) -> openai.GenericClient

Creates a client. Construction reads no environment variable, so declaring one is always safe; credentials and endpoints resolve when a request is built.

Instance methods

function

compat

(
self
) -> openai.internal.ChatCompat throws baml.errors.Io | baml.errors.ParseError | ai.errors.InvalidRequest

The provider record: everything the shared chat core needs to know about this endpoint.

function

params

(
self,
preview: bool = …
) -> openai.internal.ChatParams throws baml.errors.Io | baml.errors.ParseError

The per-request parameters handed to the shared chat core.

Parameters

  • preview: omit the credential, for rendering a request without sending it.
function

resolved_api_key

(self) -> string | null throws baml.errors.Io | baml.errors.ParseError

No canonical variable: an unauthenticated request is the normal case for a self-hosted server, so a missing key is not an error here.

function

resolved_base_url

(
self
) -> string throws baml.errors.Io | baml.errors.ParseError | ai.errors.InvalidRequest

Throws ai.errors.InvalidRequest when no base_url resolves. A base_url = env.NAME naming an UNSET variable resolves to null and lands here too, so it reports the same clear configuration error rather than a mystery request to nowhere.

Implementations

ai.Client for openai.GenericClient

Instance methods

function

id

(self) -> string
function

invoke

(
self,
input: ai.ModelTurnInput
) -> ai.ModelTurn throws baml.errors.Timeout | baml.errors.UnknownError | reflect.errors.CompilationError | ai.errors.Failure
function

render

(
self,
input: ai.ModelTurnInput
) -> baml.http.Request throws baml.errors.Timeout | baml.errors.UnknownError | reflect.errors.CompilationError | ai.errors.Failure

Source:<builtin>/openai/generic.bamlbytes 5605–6410

ai.stream.StreamingClient for openai.GenericClient

Instance methods

function

invoke_stream

(
self,
input: ai.ModelTurnInput
) -> ai.stream.TurnStream throws baml.errors.Timeout | baml.errors.UnknownError | reflect.errors.CompilationError | ai.errors.Failure

Source:<builtin>/openai/generic.bamlbytes 6416–6990