Together AI - Portkey Docs

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Together AI’s hosted models. With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

Quick Start

Get Together AI working in 3 steps:

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @together-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @together-ai provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @together-ai provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @together-ai provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

# 1. Add @together-ai provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

Tip: You can also set provider="@together-ai" in Portkey() and use just model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" in the request.

Add Provider in Model Catalog

Go to Model Catalog → Add Provider
Select Together AI
Choose existing credentials or create new by entering your Together AI API key
Name your provider (e.g., together-ai-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Reasoning / Thinking Support

Together AI supports reasoning models that expose their internal chain of thought. Use the reasoning_effort parameter to control reasoning behavior, and set strict_open_ai_compliance=False to receive the thinking content in content_blocks.

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    strict_open_ai_compliance=False
)

response = portkey.chat.completions.create(
    model="@together-ai/deepseek-ai/DeepSeek-R1",
    messages=[{"role": "user", "content": "Solve step by step: What is 23 * 47?"}],
    reasoning_effort="medium"
)

# Access thinking content from content_blocks
for block in response.choices[0].message.content_blocks:
    if block.get("type") == "thinking":
        print("Thinking:", block["thinking"])
    elif block.get("type") == "text":
        print("Response:", block["text"])

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    strictOpenAiCompliance: false
})

const response = await portkey.chat.completions.create({
    model: "@together-ai/deepseek-ai/DeepSeek-R1",
    messages: [{ role: "user", content: "Solve step by step: What is 23 * 47?" }],
    reasoning_effort: "medium"
})

// Access thinking content from content_blocks
for (const block of response.choices[0].message.content_blocks) {
    if (block.type === "thinking") {
        console.log("Thinking:", block.thinking)
    } else if (block.type === "text") {
        console.log("Response:", block.text)
    }
}

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-strict-open-ai-compliance: false" \
  -d '{
    "model": "@together-ai/deepseek-ai/DeepSeek-R1",
    "messages": [
      { "role": "user", "content": "Solve step by step: What is 23 * 47?" }
    ],
    "reasoning_effort": "medium"
  }'

The reasoning response includes content_blocks with both the model’s thinking process and the final answer. Streaming is also supported and returns reasoning chunks in the content_blocks field of the stream delta.

Thinking Mode Documentation

Learn more about thinking/reasoning support across providers

Video Generation

Together AI supports video generation through Portkey’s proxy. Generate videos from text prompts and track usage in your Portkey dashboard.

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@together-ai"
)

body = {
    "prompt": "A serene sunset over the ocean with gentle waves",
    "model": "minimax/hailuo-02",
    "width": 1366,
    "height": 768
}

# Request to generate video
response = portkey.post(
    url="/v2/videos",
    **body
)

# Request to retrieve video status
response = portkey.get(
    path="/v2/videos/<video_id>",
)

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "@together-ai"
})

const body = {
    prompt: "A serene sunset over the ocean with gentle waves",
    model: "minimax/hailuo-02",
    width: 1366,
    height: 768
}

// Request to generate video
const response = await portkey.post(
    "/v2/videos",
    body
)

// Request to retrieve video status (use the video id from the create response)
// const statusResponse = await portkey.get(`/v2/videos/${videoId}`)

curl --location 'https://api.portkey.ai/v1/v2/videos' \
  --header 'x-portkey-provider: @together-ai' \
  --header 'Content-Type: application/json' \
  --header 'x-portkey-api-key: PORTKEY_API_KEY' \
  --data '{
    "prompt": "A serene sunset over the ocean with gentle waves",
    "model": "minimax/hailuo-02",
    "width": 1366,
    "height": 768
  }'

Video generation pricing is logged in your Portkey dashboard for cost tracking.

Managing Together AI Prompts

Manage all prompt templates to Together AI in the Prompt Library. All current Together AI models are supported, and you can easily test different prompts. Use the portkey.prompts.completions.create interface to use the prompt in an application.

Next Steps

Add Metadata

Add metadata to your Together AI requests

Gateway Configs

Add gateway configs to your Together AI requests

Tracing

Trace your Together AI requests

Fallbacks

Setup fallback from OpenAI to Together AI

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation

​Quick Start

​Add Provider in Model Catalog

Complete Setup Guide →

​Reasoning / Thinking Support

Thinking Mode Documentation

​Video Generation

​Managing Together AI Prompts

​Next Steps

Add Metadata

Gateway Configs

Tracing

Fallbacks

SDK Reference

Quick Start

Add Provider in Model Catalog

Reasoning / Thinking Support

Video Generation

Managing Together AI Prompts

Next Steps