> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-chore-v2-11-2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Modal Labs

> Integrate Modal with Portkey for seamless completions and streaming.

## Quick Start

Get started with Modal in under 2 minutes:

<CodeGroup>
  ```python Python icon="python" theme={null}
  from portkey_ai import Portkey

  # 1. Install: pip install portkey-ai
  # 2. Add @modal provider in model catalog
  # 3. Use it:

  portkey = Portkey(api_key="PORTKEY_API_KEY")

  response = portkey.chat.completions.create(
      model="@modal/your-model-name",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js Javascript icon="square-js" theme={null}
  import Portkey from 'portkey-ai'

  // 1. Install: npm install portkey-ai
  // 2. Add @modal provider in model catalog
  // 3. Use it:

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY"
  })

  const response = await portkey.chat.completions.create({
      model: "@modal/your-model-name",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```python OpenAI Py icon="python" theme={null}
  from openai import OpenAI
  from portkey_ai import PORTKEY_GATEWAY_URL

  # 1. Install: pip install openai portkey-ai
  # 2. Add @modal provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # Portkey API key
      base_url=PORTKEY_GATEWAY_URL
  )

  response = client.chat.completions.create(
      model="@modal/your-model-name",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={null}
  import OpenAI from "openai"
  import { PORTKEY_GATEWAY_URL } from "portkey-ai"

  // 1. Install: npm install openai portkey-ai
  // 2. Add @modal provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // Portkey API key
      baseURL: PORTKEY_GATEWAY_URL
  })

  const response = await client.chat.completions.create({
      model: "@modal/your-model-name",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```sh cURL icon="square-terminal" theme={null}
  # 1. Add @modal provider in model catalog
  # 2. Use it:

  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -d '{
      "model": "@modal/your-model-name",
      "messages": [{"role": "user", "content": "Hello!"}]
    }'
  ```
</CodeGroup>

<Note>
  **Tip:** You can also set `provider="@modal"` in `Portkey()` and use just `model="your-model-name"` in the request.
</Note>

## Add Provider in Model Catalog

Before making requests, add Modal to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Modal**
3. Enter your [Modal API credentials](https://modal.com/)
   * Modal supports **API Key** (Bearer token) or **Custom Headers** (`modal-key` and `modal-secret`)
4. **Custom Host:** Enter your Modal deployment endpoint (e.g., `https://your-modal-deployment.modal.run`)
5. Name your provider (e.g., `modal`)

<Info>
  **Note:** Custom host configuration is set once in Model Catalog and applies to all requests using this provider.
</Info>

<Card title="Complete Setup Guide" icon="book" href="/product/model-catalog">
  See all setup options and detailed configuration instructions
</Card>

<Card title="Modal Documentation" icon="book" href="https://modal.com/docs">
  Explore the official Modal documentation
</Card>

***

## Modal Capabilities

### Streaming

Stream responses for real-time output:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY")

  stream = portkey.chat.completions.create(
      model="@modal/your-model",
      messages=[{"role": "user", "content": "Tell me a story"}],
      stream=True
  )

  for chunk in stream:
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY'
  });

  const stream = await portkey.chat.completions.create({
      model: '@modal/your-model',
      messages: [{ role: 'user', content: 'Tell me a story' }],
      stream: true
  });

  for await (const chunk of stream) {
      process.stdout.write(chunk.choices[0]?.delta?.content || '');
  }
  ```
</CodeGroup>

***

## Supported Endpoints and Parameters

| Endpoint            | Supported Parameters                                                                      |
| ------------------- | ----------------------------------------------------------------------------------------- |
| `/chat/completions` | messages, max\_tokens, temperature, top\_p, stream, presence\_penalty, frequency\_penalty |

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Gateway Configs" icon="sliders" href="/product/ai-gateway">
    Add fallbacks, load balancing, and more
  </Card>

  <Card title="Observability" icon="chart-line" href="/product/observability">
    Monitor and trace your Modal requests
  </Card>

  <Card title="Prompt Library" icon="book" href="/product/prompt-engineering-studio">
    Manage and version your prompts
  </Card>

  <Card title="Metadata" icon="tag" href="/product/observability/metadata">
    Add custom metadata to requests
  </Card>
</CardGroup>

For complete SDK documentation:

<Card title="SDK Reference" icon="code" href="/api-reference/sdk/list">
  Complete Portkey SDK documentation
</Card>