LLM Kwargs Reference¶

Per-call generation parameters forwarded from @mesh.llm consumers to the underlying vendor SDK.

Overview¶

model_params is the per-call kwarg surface that flows from a @mesh.llm consumer, through the mesh proxy, to the resolved provider, and finally into the vendor's native SDK (Anthropic, OpenAI, Gemini). Mesh does not invent its own parameter names — common knobs (max_tokens, temperature, ...) work across all three vendors; vendor-specific knobs (thinking_config, reasoning_effort, top_k, ...) unlock per-vendor features.

The exact passthrough surface for each Python adapter lives in _<VENDOR>_PASSTHROUGH_KWARGS (see Source pointers below). Anything outside that set is logged as a once-per-key WARN by the adapter so a typo or a newer litellm-only knob surfaces immediately instead of being silently dropped.

Common kwargs (cross-vendor)¶

These work across Anthropic, OpenAI, and Gemini:

Mesh forwards these unchanged for OpenAI and Gemini. For Anthropic, mesh translates stop → stop_sequences to match the native SDK.

Vendor-specific kwargs¶

Anthropic-only¶

OpenAI-only¶

Gemini-only¶

Per-language usage¶

Python¶

model_params={...} is a parameter on @mesh.llm. Values are merged into the MeshLlmRequest.model_params dict at runtime and forwarded to the resolved provider.

import mesh

@mesh.llm(
    provider={"capability": "llm", "tags": ["+gemini"]},
    model_params={
        "max_tokens": 4096,
        "temperature": 0.3,
        "thinking_config": {"thinking_budget": 0},  # disable thinking for fast Gemini 2.5
    },
)
async def my_tool(prompt: str, llm: mesh.MeshLlmAgent = None) -> str:
    return await llm(prompt)

See MeshLlmRequest for the underlying request shape.

TypeScript¶

The TS SDK exposes a typed options surface on MeshLlmAgentConfig. Options are mapped to model_params keys on the wire (e.g. maxOutputTokens → max_tokens, topP → top_p).

import { MeshLlmAgent } from "@mcpmesh/sdk";

const agent = new MeshLlmAgent({
  functionId: "my_tool",
  provider: { capability: "llm", tags: ["+claude"] },
  model: "anthropic/claude-sonnet-4-5",
  maxIterations: 5,
  maxOutputTokens: 4096,
  temperature: 0.3,
  topP: 0.95,
  stop: ["\n\n---"],
  parallelToolCalls: true,
});

const result = await agent.run("Help me draft a release note", {
  tools: resolvedToolProxies,
  meshProvider: { endpoint: "http://provider:9000", functionName: "process_chat" },
});

For vendor-specific kwargs the typed surface doesn't expose (e.g. Gemini thinking_config, Anthropic output_config, OpenAI reasoning_effort), use the modelParams escape hatch on LlmCallOptions. The dict is merged into the wire model_params before typed fields, so typed options (maxOutputTokens, temperature, ...) win on collision and remain authoritative:

const reply = await llm.call(prompt, {
  maxOutputTokens: 4096,
  temperature: 0.3,
  modelParams: {
    thinking_config: { thinking_budget: 0 }, // escape hatch for vendor-specific kwargs
  },
});

Source: src/runtime/typescript/src/llm-agent.ts.

Java¶

The Java SDK uses the @MeshLlm annotation for tool-call defaults and a fluent builder on MeshLlmAgent for per-call overrides. Builder values are translated into model_params keys on the wire (maxTokens → max_tokens, topP → top_p).

import io.mcpmesh.types.MeshLlmAgent;
import io.mcpmesh.types.annotations.MeshLlm;
import io.mcpmesh.types.annotations.MeshTool;

@MeshLlm(
    providerSelector = "capability=llm,tags=+claude",
    maxTokens = 4096,
    temperature = 0.3
)
@MeshTool(capability = "summarizer")
public String summarize(@Param("text") String text, MeshLlmAgent llm) {
    return llm.request()
        .system("Summarize the following text in 2 sentences.")
        .user(text)
        .maxTokens(1024)        // override the annotation default for this call
        .temperature(0.5)
        .topP(0.95)
        .stop("END")
        .generate();
}

For vendor-specific kwargs the typed builder doesn't expose (e.g. Gemini thinking_config, Anthropic output_config, OpenAI reasoning_effort), use the .modelParams(...) escape hatch on the builder. The map is merged into the wire model_params before typed setters, so typed setters (maxTokens, temperature, ...) win on collision and remain authoritative:

String response = llm.request()
    .user(prompt)
    .maxTokens(4096)
    .temperature(0.3)
    .modelParams(Map.of(
        "thinking_config", Map.of("thinking_budget", 0)
    ))
    .generate();

The same builder surface — including .modelParams(...) — is available on the streaming path via .streamGenerate(). The merge semantics (escape hatch first, typed setters win, annotation defaults only when unset) are shared with the buffered .generate() path:

Flow.Publisher<String> chunks = llm.request()
    .system("You are helpful")
    .user(prompt)
    .maxTokens(4096)
    .temperature(0.7)
    .modelParams(Map.of(
        "thinking_config", Map.of("thinking_budget", 0)
    ))
    .streamGenerate();

Streaming requires the consumer to opt in via the ai.mcpmesh.stream tag on @MeshLlm(providerSelector = ...) — see Java LLM Integration.

Source: MeshLlmAgentProxy.java.

Reference matrix¶

Source pointers¶

For the precise passthrough surface per adapter, see the _<VENDOR>_PASSTHROUGH_KWARGS frozenset at the top of each native client:

• Anthropic: src/runtime/python/_mcp_mesh/engine/native_clients/anthropic_native.py
• OpenAI: src/runtime/python/_mcp_mesh/engine/native_clients/openai_native.py
• Gemini: src/runtime/python/_mcp_mesh/engine/native_clients/gemini_native.py

Any kwarg outside the passthrough + handled sets emits a once-per-key WARN — useful diagnostic surface for callers debugging cross-vendor passthrough.

See also¶

• Environment Variables - configure provider API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY) and runtime overrides (MESH_LLM_MODEL, MESH_LLM_MAX_ITERATIONS).
• Python LLM Integration - @mesh.llm consumer guide.
• Java LLM Integration - @MeshLlm consumer guide.
• TypeScript LLM Integration - LLM agent consumer guide.