Supported LLM Providers

Overview

Archestra Platform acts as a security proxy between your AI applications and LLM providers. It currently supports the following LLM providers.

OpenAI

Supported OpenAI APIs

Chat Completions API (/chat/completions) - ✅ Fully supported
Responses API (/responses) - ⚠️ Not yet supported (GitHub Issue #720)

OpenAI Connection Details

Base URL: http://localhost:9000/v1/openai/{agent-id}
Authentication: Pass your OpenAI API key in the Authorization header as Bearer <your-api-key>

Important Notes

Use Chat Completions API: Ensure your application uses the /chat/completions endpoint (not /responses). Many frameworks default to this, but some like Vercel AI SDK require explicit configuration (add .chat to the provider instance).
Streaming: OpenAI streaming responses require your cloud provider's load balancer to support long-lived connections. See Cloud Provider Configuration for more details.

Anthropic

Supported Anthropic APIs

Messages API (/messages) - ✅ Fully supported

Anthropic Connection Details

Base URL: http://localhost:9000/v1/anthropic/{agent-id}
Authentication: Pass your Anthropic API key in the x-api-key header

Google Gemini

Archestra supports both the Google AI Studio (Gemini Developer API) and Vertex AI implementations of the Gemini API.

Supported Gemini APIs

Generate Content API (:generateContent) - ✅ Fully supported
Stream Generate Content API (:streamGenerateContent) - ✅ Fully supported

Gemini Connection Details

Base URL: http://localhost:9000/v1/gemini/{agent-id}/v1beta
Authentication:
- Google AI Studio (default): Pass your Gemini API key in the x-goog-api-key header
- Vertex AI: No API key required from clients - uses server-side Application Default Credentials (ADC)

Important Notes

API Version: Archestra uses the v1beta Gemini API version for feature parity with the latest Gemini capabilities.
Tool Support: Function calling (tool use) is fully supported, including tool invocation policies and trusted data policies.

Using Vertex AI

To use Vertex AI instead of Google AI Studio, configure these environment variables:

Variable	Required	Description
`ARCHESTRA_GEMINI_VERTEX_AI_ENABLED`	Yes	Set to `true` to enable Vertex AI mode
`ARCHESTRA_GEMINI_VERTEX_AI_PROJECT`	Yes	Your GCP project ID
`ARCHESTRA_GEMINI_VERTEX_AI_LOCATION`	No	GCP region (default: `us-central1`)
`ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE`	No	Path to service account JSON key file

GKE with Workload Identity (Recommended)

For GKE deployments, we recommend using Workload Identity which provides secure, keyless authentication. This eliminates the need for service account JSON key files.

Setup steps:

Create a GCP service account with Vertex AI permissions:

gcloud iam service-accounts create archestra-vertex-ai \
  --display-name="Archestra Vertex AI"

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Bind the GCP service account to the Kubernetes service account:

gcloud iam service-accounts add-iam-policy-binding \
  archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"

Replace NAMESPACE with your Helm release namespace and KSA_NAME with the Kubernetes service account name (defaults to archestra-platform).

Configure Helm values to annotate the service account:

archestra:
  orchestrator:
    kubernetes:
      serviceAccount:
        annotations:
          iam.gke.io/gcp-service-account: archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com
  env:
    ARCHESTRA_GEMINI_VERTEX_AI_ENABLED: "true"
    ARCHESTRA_GEMINI_VERTEX_AI_PROJECT: "PROJECT_ID"
    ARCHESTRA_GEMINI_VERTEX_AI_LOCATION: "us-central1"

With this configuration, Application Default Credentials (ADC) will automatically use the bound GCP service account—no credentials file needed.

Other Environments

For non-GKE environments or when Workload Identity isn't available, set ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE to the path of a service account JSON key file.