JavaScript (Node.js) Quick Start Guide for v1-chat-completions API

Last updated January 24, 2025

This article is a work in progress, or documents a feature that is not yet released to all users. This article is unlisted. Only those with the link can access it.

The Heroku Managed Inference and Agent add-on is currently in pilot. The products offered as part of the pilot aren’t intended for production use and are considered as a Beta Service and are subject to the Beta Services terms at https://www.salesforce.com/company/legal/agreements.jsp.

Our Claude chat models (Claude 3.5 Sonnet Latest, Claude 3.5 Sonnet, Claude 3.5 Haiku,and Claude 3.0 Haiku) generate conversational completions for input messages. This guide walks you through how to use the v1-chat-completions API with JavaScript.

Prerequisites

Before making requests, provision access to the model of your choice.

  1. If it’s not already installed, install the Heroku CLI. Then install the Heroku AI plugin:

    heroku plugins:install @heroku/plugin-ai

  2. Attach a chat model to an app of yours:

    # If you don't have an app yet, you can create one with:
heroku create $APP_NAME # specify the name you want for your app (or skip this step to use an existing app you have)

# Create and attach one of our chat models to your app, $APP_NAME:
heroku ai:models:create -a $APP_NAME claude-3-5-sonnet --as INFERENCE
# OR
heroku ai:models:create -a $APP_NAME claude-3-haiku --as INFERENCE

  3. Install the necessary axios package:

    npm install axios

JavaScript Example Code

const axios = require('axios');

// Assert that environment variables are set
const INFERENCE_URL = process.env.INFERENCE_URL;
const INFERENCE_KEY = process.env.INFERENCE_KEY;
const INFERENCE_MODEL_ID = process.env.INFERENCE_MODEL_ID;

if (!INFERENCE_URL || !INFERENCE_KEY || !INFERENCE_MODEL_ID) {
    console.error("Missing required environment variables.");
    console.log("Set them up using the following commands:");
    console.log("export INFERENCE_URL=$(heroku config:get -a $APP_NAME INFERENCE_URL)");
    console.log("export INFERENCE_KEY=$(heroku config:get -a $APP_NAME INFERENCE_KEY)");
    console.log("export INFERENCE_MODEL_ID=$(heroku config:get -a $APP_NAME INFERENCE_MODEL_ID)");
    process.exit(1);
}

async function parseChatOutput(response) {
    if (response.status === 200) {
        console.log("Chat Completion:", response.data.choices[0].message.content);
    } else {
        console.log(`Request failed: ${response.status}, ${response.statusText}`);
    }
}

async function generateChatCompletion(payload) {
    try {
        const response = await axios.post(`${INFERENCE_URL}/v1/chat/completions`, payload, {
            headers: {
                'Authorization': `Bearer ${INFERENCE_KEY}`,
                'Content-Type': 'application/json'
            }
        });
        await parseChatOutput(response);
    } catch (error) {
        console.error("Error generating chat completion:", error.message);
    }
}

// Example payload
const payload = {
    model: INFERENCE_MODEL_ID,
    messages: [
        { role: "user", content: "Hello!" },
        { role: "assistant", content: "Hi there! How can I assist you today?" },
        { role: "user", content: "Why is Heroku so cool?" }
    ],
    temperature: 0.5,
    max_tokens: 100,
    stream: false
};

// Generate chat completion
generateChatCompletion(payload);

Feedback

Log in to submit feedback.