AI SDK Integration

Table of Contents [expand]

Supported Models
Installation
Prerequisites
Set Up the Integration
Configure the Integration
Using the Integration
Troubleshooting Common Issues
Additional Resources

Last updated November 19, 2025

Use Heroku’s AI inference services through Vercel’s AI SDK, an open-source library for building AI-powered applications and agents. You can build AI features like chatbots, semantic search, and image generation while keeping your code consistent across providers.

The heroku-ai-provider integration lets you call Heroku-hosted models in your Vercel applications using the same SDK functions you use with other providers. The integration works with Next.js, Express.js, and other frameworks that support the Vercel AI SDK.

Supported Models

To view a list of chat, embedding, and image models you can use with Vercel, see Managed Inference and Agents API Model Cards.

Installation

Install the integration with your preferred package manager:

npm:
```
npm install heroku-ai-provider
```
pnpm:
```
pnpm add heroku-ai-provider
```
yarn:
```
yarn add heroku-ai-provider
```

Prerequisites

A Heroku account with access to AI services
Heroku AI API keys for inference and/or embeddings
The Vercel AI SDK installed in your project
```
npm install ai
```

Set Up the Integration

Create an app in Heroku:
```
heroku create example-app
```

Create and attach an AI model to your app:

heroku ai:models:create -a example-app claude-3-5-haiku

Export environment variables:

# for chat completions
export INFERENCE_KEY=$(heroku config:get INFERENCE_KEY -a example-app)

# for embeddings
export EMBEDDING_KEY=$(heroku config:get EMBEDDING_KEY -a example-app)

# for image generation
export DIFFUSION_KEY=$(heroku config:get DIFFUSION_KEY -a example-app)

# (optional) custom API endpoints
export INFERENCE_URL=https://us.inference.heroku.com
export EMBEDDING_URL=https://us.inference.heroku.com
export DIFFUSION_URL=https://us.inference.heroku.com

Configure the Integration

import { heroku } from "heroku-ai-provider";
const model = heroku.chat("example-model");

Custom Configuration

import { createHerokuAI } from "heroku-ai-provider";

const client = createHerokuAI({
  chatApiKey: "INFERENCE_KEY",
  embeddingsApiKey: "EMBEDDING_KEY",
  chatBaseUrl: "https://us.inference.heroku.com/v1/chat/completions",
  embeddingsBaseUrl: "https://us.inference.heroku.com/v1/embeddings",
  imageApiKey: "DIFFUSION_KEY",
  imageBaseUrl: "https://us.inference.heroku.com/v1/images/generations",
});

Using the Integration

You can use Heroku language models with various AI SDK functions, including generateText, streamText, tool calling, embeddings, and image generation.

Chat Completions

Basic Chat Example

import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt: "What is the capital of France?",
});

console.log(text); // "The capital of France is Paris."

Streaming Chat Example

import { streamText, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";

const { textStream } = await streamText({
  model: heroku.chat("claude-3-haiku"),
  prompt: "Write a short story about a robot learning to paint.",
});

for await (const delta of textStream) {
  process.stdout.write(delta);
}

System Message Chat Example

import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  system: "You are a helpful assistant that explains complex topics simply.",
  prompt: "Explain quantum computing",
});

Tool Usage

Basic Tool Usage Example

import { generateText, tool, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt: "What is the weather like in New York?",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      parameters: z.object({
        location: z.string().describe("The city and state"),
      }),
      execute: async ({ location }) => {
        // Simulate weather API call
        return {
          location,
          temperature: 72,
          condition: "sunny",
        };
      },
    }),
  },
  stopWhen: stepCountIs(5),
});

Advanced Tool Usage with Multiple Steps Example

import { generateText, tool, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

const { text, steps } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt:
    "Check the weather in New York and then suggest appropriate clothing.",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      parameters: z.object({
        location: z.string().describe("The city and state"),
      }),
      execute: async ({ location }) => {
        return {
          location,
          temperature: 45,
          condition: "rainy",
          humidity: 80,
        };
      },
    }),
    suggestClothing: tool({
      description: "Suggest appropriate clothing based on weather conditions",
      inputSchema: z.object({
        temperature: z.number().describe("Temperature in Fahrenheit"),
        condition: z.string().describe("Weather condition"),
        humidity: z.number().optional().describe("Humidity percentage"),
      }),
      execute: async ({ temperature, condition, humidity }) => {
        return {
          suggestions: [
            "Waterproof jacket",
            "Warm layers",
            "Waterproof shoes",
            "Umbrella",
          ],
          reasoning: `Given ${temperature}°F and ${condition} weather${humidity ? ` with ${humidity}% humidity` : ""}, you'll want to stay warm and dry.`,
        };
      },
    }),
  },
  stopWhen: stepCountIs(5),
});

console.log("Final response:", text);
console.log("Tool execution steps:", steps.length);

Embeddings

Basic Embeddings Example

import { embed } from "ai";
import { heroku } from "heroku-ai-provider";

const { embedding } = await embed({
  model: heroku.embedding("cohere-embed-multilingual"),
  value: "Hello, world!",
});

console.log(embedding); // [0.1, 0.2, -0.3, ...]

Multiple Embeddings Example

import { embedMany } from "ai";
import { heroku } from "heroku-ai-provider";

const { embeddings } = await embedMany({
  model: heroku.embedding("cohere-embed-multilingual"),
  values: ["First document", "Second document", "Third document"],
});

console.log(embeddings.length); // 3

Convenience Function Example

import { createEmbedFunction } from "heroku-ai-provider";

// Create a reusable embed function
const embedText = createEmbedFunction({
  apiKey: process.env.EMBEDDING_KEY!,
  model: "cohere-embed-multilingual",
});

const embedding = await embedText("Hello, world!");
console.log(embedding); // [0.1, 0.2, -0.3, ...]

Image Generation

Image Generation Example

import { experimental_generateImage as generateImage } from "ai";
import { heroku } from "heroku-ai-provider";

const result = await generateImage({
  model: heroku.image("stable-image-ultra"),
  prompt: "A watercolor illustration of a lighthouse at sunrise",
  size: "1024x1024",
});

const imageBytes = result.image.uint8Array;
console.log("Generated image bytes length:", imageBytes.length);

Configuration Options

HerokuProviderSettings

interface HerokuProviderSettings {
  // API keys (falls back to environment variables)
  chatApiKey?: string; // INFERENCE_KEY
  embeddingsApiKey?: string; // EMBEDDING_KEY
  imageApiKey?: string; // DIFFUSION_KEY

  // Base URLs (falls back to environment variables or defaults)
  chatBaseUrl?: string; // INFERENCE_URL
  embeddingsBaseUrl?: string; // EMBEDDING_URL
  imageBaseUrl?: string; // DIFFUSION_URL
}

Framework Integration

Next.js App Router Example

// app/api/chat/route.ts
import { streamText, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: heroku.chat("claude-4-sonnet"),
    messages,
    stopWhen: stepCountIs(5), // Enable multi-step tool conversations
  });

  return result.toDataStreamResponse();
}

Next.js with Tool Support Example

// app/api/chat/route.ts
import { streamText, tool } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: heroku.chat("claude-4-sonnet"),
    messages,
    tools: {
      getTime: tool({
        description: "Get the current time",
        inputSchema: z.object({
          timezone: z
            .string()
            .optional()
            .describe("Timezone (e.g., 'America/New_York')"),
        }),
        execute: async ({ timezone = "UTC" }) => {
          return {
            time: new Date().toLocaleString("en-US", { timeZone: timezone }),
            timezone,
          };
        },
      }),
    },
    stopWhen: stepCountIs(5),
  });

  return result.toDataStreamResponse();
}

Express.js Example

import express from "express";
import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const app = express();

app.post("/chat", async (req, res) => {
  const { prompt } = req.body;

  const { text } = await generateText({
    model: heroku.chat("claude-3-haiku"),
    prompt,
  });

  res.json({ response: text });
});

Error Handling

The integration enables comprehensive error handling with user-friendly error messages:

import {
  createHerokuAI,
  isConfigurationError,
  isTemporaryServiceError,
} from "heroku-ai-provider";

try {
  const result = await generateText({
    model: heroku.chat("claude-4-sonnet"),
    prompt: "Hello!",
  });
} catch (error) {
  if (isConfigurationError(error)) {
    console.error("Configuration error:", error.message);
    // Handle API key or URL configuration issues
  } else if (isTemporaryServiceError(error)) {
    console.error("Service error:", error.message);
    // Handle temporary service issues (retry logic)
  } else {
    console.error("Unexpected error:", error);
  }
}

Troubleshooting Common Issues

Authentication Errors

If you get the error “Chat API key is required” or “Embeddings API key is required”, ensure your API keys are set in environment variables or passed directly to createHerokuAI().

Model Not Found

If you get the error “Unsupported chat model” or “Unsupported embedding model”, confirm that you’re using a supported model.

Network Errors

If you experience connection timeouts or network failures, verify your internet connection and that Heroku’s AI services are accessible.

URL Configuration

If you get the error “Invalid URL format”, ensure custom URLs are valid and use HTTP/HTTPS protocol.

Tool Execution Issues

If you call tools and the AI isn’t providing a final response, ensure you configure stopWhen (for example, stopWhen: stepCountIs(5)) so the model can complete multi-step tool conversations.

Schema Validation Errors

This provider automatically filters out problematic schema properties (for example, $schema) that some validation libraries add.

Additional Resources

Keep Reading

Or explore the AI Integrations category.

Categories