Llama Agents

Add observability, reliability, and multi-provider support to Llama Agents.

Portkey enhances Llama Agents with production features:

• Complete observability of agent steps and LLM interactions
• Built-in reliability with fallbacks, retries, and load balancing
• Access to 1600+ LLMs through a single integration
• Cost tracking and optimization

Quick Start

from llama_index.llms.openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

llm = OpenAI(
    api_base=PORTKEY_GATEWAY_URL,
    api_key="YOUR_PORTKEY_API_KEY",
    default_headers=createHeaders(provider="@openai-prod")
)

All agent interactions are now logged in your Portkey dashboard.

Setup

1. Install packages

   pip install -qU llama-agents llama-index portkey-ai
   
2. Add provider in Model Catalog

   Go to Model Catalog → Add Provider. Select your provider, enter API keys, and name it (e.g., openai-prod).

   Your provider slug is @openai-prod.

3. Get Portkey API Key

   Create an API key at app.portkey.ai/api-keys.

   Pro tip: Attach a default config for fallbacks and caching.

4. Configure LlamaIndex LLM

   from llama_index.llms.openai import OpenAI
   from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
   
   llm = OpenAI(
       api_base=PORTKEY_GATEWAY_URL,
       api_key="YOUR_PORTKEY_API_KEY",
       default_headers=createHeaders(provider="@openai-prod")
   )
   

Production Features

Observability

Add trace IDs and metadata for filtering:

default_headers=createHeaders(
    provider="@openai-prod",
    trace_id="research-agent-1",
    metadata={"agent_type": "research", "_user": "user_123"}
)

Tracing with Callback Handler

Use the Portkey callback handler for detailed traces:

from portkey_ai.llamaindex import LlamaIndexCallbackHandler

portkey_handler = LlamaIndexCallbackHandler(
    api_key="YOUR_PORTKEY_API_KEY",
    metadata={"session_id": "session_1", "agent_id": "research_agent_1"}
)

llm = OpenAI(
    api_key="YOUR_OPENAI_API_KEY",
    callbacks=[portkey_handler]
)

Reliability

Enable fallbacks via Configs:

{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "override_params": { "model": "@openai-prod/gpt-4o" } },
    { "override_params": { "model": "@anthropic-prod/claude-sonnet-4" } }
  ]
}

Caching

Reduce costs with response caching:

{
  "cache": { "mode": "semantic" }
}

Switching Providers

Change the provider to switch models:

# OpenAI
createHeaders(provider="@openai-prod")

# Anthropic
createHeaders(provider="@anthropic-prod")

# Azure OpenAI
createHeaders(provider="@azure-prod")

Supported Providers

See all 1600+ supported models

Next Steps

Model Catalog

Set up providers and budgets

Configs

Fallbacks, caching, and routing