Récif
Récifdocs

v0.1.0 · Apache 2.0

Search docs...

LLM Providers

Configure LLM providers securely — Kubernetes Secrets, Service Accounts, and Workload Identity.

6 min read

Overview

Récif supports 7 LLM providers. Credentials are never stored in the agent configuration — they live in Kubernetes Secrets and are injected at runtime by the operator.

ProviderModel TypeAuth MethodExample Models
OllamaollamaNone (local)qwen3.5:4b, llama3.1:8b, mistral:7b
OpenAIopenaiK8s Secret (API key)gpt-4o, gpt-4o-mini, o3-mini
AnthropicanthropicK8s Secret (API key)claude-sonnet-4, claude-haiku
Google AIgoogle-aiK8s Secret (API key)gemini-2.5-flash, gemini-2.5-pro
Vertex AIvertex-aiService Account (JSON key)gemini-2.5-flash, gemini-2.5-pro
AWS BedrockbedrockK8s Secret (IAM credentials)anthropic.claude-sonnet-4, amazon.titan
StubstubNone (testing)stub-echo

Credential Strategy

Récif uses Kubernetes Secrets for all credentials. Never put API keys in the Agent CRD, ConfigMaps, or environment variables directly.

┌─────────────────┐     ┌──────────────┐     ┌─────────────────┐
│  K8s Secret     │────→│   Operator   │────→│  Agent Pod      │
│  (your creds)   │     │  (injects)   │     │  (env vars set) │
└─────────────────┘     └──────────────┘     └─────────────────┘

The agent CRD references secrets by name via spec.envSecrets. The operator mounts them as environment variables. The agent code never sees the raw secret — it just reads standard env vars like OPENAI_API_KEY.

Provider Setup

Ollama (Local — No Credentials)

Ollama is included in the Helm chart by default. No API key, no setup — models run on your cluster.

# Agent CRD
apiVersion: agents.recif.dev/v1
kind: Agent
metadata:
  name: my-agent
  namespace: team-default
spec:
  modelType: ollama
  modelId: qwen3.5:4b
  # No envSecrets needed

Tip

Ollama is the fastest way to get started. Zero cost, zero credentials, works offline. Good for development and air-gapped environments.

OpenAI

Step 1: Create a Kubernetes Secret

kubectl create secret generic openai-credentials \
  --namespace team-default \
  --from-literal=OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxx

Step 2: Reference in Agent CRD

apiVersion: agents.recif.dev/v1
kind: Agent
metadata:
  name: my-agent
  namespace: team-default
spec:
  modelType: openai
  modelId: gpt-4o-mini
  envSecrets:
    - openai-credentials

Available models: gpt-4o, gpt-4o-mini, o1, o3-mini

Note

The Secret must be in thesame namespaceas the agent. For multi-team setups, each team creates their own secret in their namespace.

Anthropic

Step 1: Create Secret

kubectl create secret generic anthropic-credentials \
  --namespace team-default \
  --from-literal=ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxxx

Step 2: Reference in Agent CRD

spec:
  modelType: anthropic
  modelId: claude-sonnet-4-20250514
  envSecrets:
    - anthropic-credentials

Available models: claude-sonnet-4-20250514, claude-haiku-4-5-20251001

Google AI (API Key)

The simplest way to use Gemini models. Get a free API key from Google AI Studio.

Step 1: Create Secret

kubectl create secret generic google-ai-credentials \
  --namespace team-default \
  --from-literal=GOOGLE_API_KEY=AIzaSyxxxxxxxxxxxxx

Step 2: Reference in Agent CRD

spec:
  modelType: google-ai
  modelId: gemini-2.5-flash
  envSecrets:
    - google-ai-credentials

Warning

Google AI (API key) is for development and prototyping. For production with billing controls, audit logs, and VPC, useVertex AIwith a Service Account instead.

Vertex AI (Service Account — Production)

Vertex AI uses a GCP Service Account for authentication. This provides per-agent isolation, audit trails, and fine-grained IAM permissions.

Step 1: Create a GCP Service Account

# Create the service account
gcloud iam service-accounts create hr-agent \
  --project=my-project-123 \
  --display-name="HR Agent"
 
# Grant Vertex AI permissions
gcloud projects add-iam-policy-binding my-project-123 \
  --member="serviceAccount:hr-agent@my-project-123.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"
 
# Download the key
gcloud iam service-accounts keys create hr-agent-key.json \
  --iam-account=hr-agent@my-project-123.iam.gserviceaccount.com

Step 2: Create K8s Secret from the key

kubectl create secret generic hr-agent-gcp-sa \
  --namespace team-default \
  --from-file=credentials.json=hr-agent-key.json

Step 3: Reference in Agent CRD

apiVersion: agents.recif.dev/v1
kind: Agent
metadata:
  name: hr-assistant
  namespace: team-default
spec:
  modelType: vertex-ai
  modelId: gemini-2.5-flash
  gcpServiceAccount: "hr-agent@my-project-123.iam.gserviceaccount.com"

The operator automatically:

  • Mounts hr-assistant-gcp-sa secret into the pod
  • Sets GOOGLE_APPLICATION_CREDENTIALS to the mounted file
  • Extracts GOOGLE_CLOUD_PROJECT from the service account email

Step 4 (GKE only): Use Workload Identity instead of key files

On GKE, you can avoid key files entirely with Workload Identity:

# Link K8s service account to GCP service account
gcloud iam service-accounts add-iam-policy-binding \
  hr-agent@my-project-123.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:my-project-123.svc.id.goog[team-default/hr-assistant]"

No secret file needed — the pod gets GCP credentials automatically from the metadata server.

AWS Bedrock (IAM Credentials)

Step 1: Create Secret with IAM credentials

kubectl create secret generic bedrock-credentials \
  --namespace team-default \
  --from-literal=AWS_ACCESS_KEY_ID=AKIAxxxxxxxxxxxxx \
  --from-literal=AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxx \
  --from-literal=AWS_REGION=us-east-1

Step 2: Reference in Agent CRD

spec:
  modelType: bedrock
  modelId: anthropic.claude-sonnet-4-20250514-v1:0
  envSecrets:
    - bedrock-credentials

EKS alternative: Use IRSA (IAM Roles for Service Accounts)

# Annotate the K8s service account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bedrock-agent
  namespace: team-default
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/bedrock-access

No credentials secret needed — EKS injects temporary credentials automatically.

Stub (Testing)

Zero configuration. Echoes input back. Useful for testing tools, skills, and pipelines without LLM costs.

spec:
  modelType: stub
  modelId: stub-echo

Switching Providers

Change modelType and modelId in the CRD — the operator handles the rest:

# Before: OpenAI
spec:
  modelType: openai
  modelId: gpt-4o-mini
  envSecrets: [openai-credentials]
 
# After: Ollama (local, free)
spec:
  modelType: ollama
  modelId: qwen3.5:4b
  envSecrets: []
kubectl apply -f my-agent.yaml
# Operator rolling-updates the pod — zero downtime

Security Best Practices

PracticeWhy
One Secret per provider per namespaceTeam isolation — team A's OpenAI key doesn't leak to team B
Service Accounts for GCP/AWSPer-agent permissions, audit trails, revocable
Workload Identity on GKE/EKSZero secrets to manage, automatic rotation
Never put keys in CRD or ConfigMapCRDs are visible to anyone with K8s read access
Rotate secrets regularlyUse kubectl create secret --dry-run=client -o yaml | kubectl apply -f - to update without downtime
Scope permissions minimallyGCP SA should only have roles/aiplatform.user, not roles/editor

Credential Flow

1. Admin creates K8s Secret (API key or SA key file)
              │
              ▼
2. Agent CRD references secret via spec.envSecrets
              │
              ▼
3. Operator sees the CRD, builds Deployment with envFrom secretRef
              │
              ▼
4. Pod starts → env vars are injected (OPENAI_API_KEY, etc.)
              │
              ▼
5. Corail reads env vars → connects to LLM provider

The agent code never touches secrets directly. Kubernetes handles injection, the operator handles wiring, and RBAC controls who can read which secrets.